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Dr.  Alan  E.  Diehl 


HJ»§.  Air  F©rc®  Safety  Agency 


DOES  COCKPIT  MANAGEMENT  TRAINING 
REDUCE  AIRCREW  ERROR? 


Conclusion: 

The  results  of  six  empirical  and  six  operational  evaluations 
provide  strong  evidence  that  these  training  programs  can  help 
reduce  aircrew  errors  and  thereby  prevent  accidents. 
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Paper  Presented  at  the  22nd  International  Seminar 
International  Society  of  Air  Safety  Investigators 
Canberra  Australia,  November  1991 


DOES  COCKPIT  MANAGEMENT  TRAINING 
REDUCE  AIRCREW  ERROR? 


Alan  E.  Diehl,  Ph.D. 

Technical  Advisor  for  Human  Performance 
US  Air  Force  Safety  Agency 
Norton  AFB,  California 


Introduction 

Human  factors  problems  continue  to  be  involved  in  majority  of 
mishaps.  Thus,  the  causes  and  cures  for  aircrew  "error"  are 
widely  discussed  topics.  Several  experts  (e.g.  Bruggink,  1978, 
Miller,  1979  and  Nance,  1986)  have  noted  that  labels  like  pilot 
"error"  are  often  misapplied  in  describing  ergonomic,  management, 
regulatory  or  systems  design  shortcomings.  This  paper  uses  the 
generic  term  "cockpit  management"  when  referring  to  the  wide 
variety  of  programs  which  are  designed  to  reduce  aircrew  errors. 
In  fact,  recent  evidence  suggests  Cockpit  Resource  Management 
(CRM)  and  Aeronautical  Decisionmaking  (ADM)  training  may  help 
reduce  aircrew  error  accident  rates  by  as  much  as  81  percent. 

These  programs  have  only  emerged  in  the  last  decade  largely 
because  of  the  fundamental  problems  associated  with  detecting  and 
controlling  human  error.  Many  of  us  have  lamented  the  greater 
difficulties  of  accurately  documenting  human  vis-a-via  mechanical 
failures,  (e.g.  mental  fatigue  is  usually  tougher  to  prove  than 
metal  fatigue).  It  is  also  often  harder  for  us  as  air  safety 
investigators  to  specify  effective  countermeasures  in  the  human 
factors  domain.  Thus,  some  organizations  unfortunately  have 
assumed  such  errors  are  "the  price  of  doing  business." 

In  these  times  of  tight  budgets,  management  and  government 
authorities  can  be  expected  to  demand  proof  that  preventive 
measures  are  in  fact  working.  Here  again,  proving  the  effective¬ 
ness  of  human  factors  initiatives  is  very  difficult,  (e.g.  crews 
can  always  comply  with  unpopular  standard  operating  procedures 
during  check-rides).  Lastly,  experts  such  as  Dr.  Clay  Foushee 
(1987)  have  aptly  noted  that  accidents,  because  of  their  relative 
infrequency,  make  poor  scientific  criteria.  It  is  also  axiomatic 
that  proving  the  negative  (accidents  which  were  prevented)  is 
even  more  difficult. 
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This  paper  addresses  these  issues  by:  1)  Examining  the  prevalence 
of  major  types  of  contemporary  errors  and  reviewing  the  tradi¬ 
tional  methods  which  have  been  used  to  improve  human  reliability, 
2)  Discussing  how  innovative  cockpit  management  training  programs 
were  developed  and  implemented,  and  3)  Describing  the  current 
evidence  on  the  effectiveness  of  such  programs. 


Taxonomy  of  Errors 

In  their  classic  study,  Jensen  and  Benel  (1977)  noted  all  aircrew 
errors  could  be  classified  into  one  of  three  major  categories 
based  on  behavioral  activities:  Procedural,  Perceptualmotor ,  and 
Decisional  Tasks.  Examples  of  procedural  tasks  include  manage¬ 
ment  of  vehicle  subsystems  and  configuration,  while  related 
errors  would  include  retracting  the  landing  gear  instead  of  flaps 
or  overlooking  checklist  items.  Perceptualmotor  tasks  include 
manipulating  flight  controls  and  throttles,  while  errors  would 
include  over  shooting  a  glide-slope  indication  or  stalling  the 
aircraft.  Decisional  tasks  include  flight  planning  and  in-flight 
hazard  evaluation,  while  errors  would  include  failing  to  delegate 
tasks  in  an  emergency  situation  or  continuing  flight  into  adverse 
weather.  These  researchers  also  noted  while  the  term  "judgment" 
is  sometimes  used  with  both  perceptualmotor  and  decisional 
processes,  it  should  be  more  closely  associated  with  the  complex 
cognitive  processes  involved  in  human  decisionmaking. 

They  analyzed  all  US  general  aviation  accidents  occurring  from 
1970  to  1974  using  the  National  Transportation  Safety  Board 
( NTSB )  computerized  data  base.  Their  analysis  of  the  fatal 
accidents  involving  pilot  error  indicated  that  264  were  procedu¬ 
ral,  2496  were  perceptualmotor,  and  2940  were  decisional  in 
nature. 

My  recent  paper  (Diehl,  1991b)  analyzed  military  and  airline 
accident  data,  for  comparison  purposes,  using  the  Jensen  and 
Benel  (1977)  taxonomy.  The  NTSB  computerized  accident  data  base 
v.'s  examined  for  US  airline  (and  scheduled  airtaxi)  accidents 
occurring  during  1987,  1988,  and  1989.  This  data  indicated  that 
24  of  the  28  major  accidents  (those  resulting  in  destroyed  air¬ 
craft  and/or  fatalities)  involved  aircrew  error.  In  these 
accidents  there  were  16  procedural,  21  perceptualmotor,  and  48 
decisional  errors.  The  relative  percentages  of  these  errors  are 
depicted  in  Table  1,  along  with  the  previously  discussed  general 
aviation  data  and  tha.  of  another  study  involving  military 
accidents. 

The  computerized  data  base  was  examined  for  US  Air  Force  (USAF) 
Class  A  flight  mishaps.  These  were  mishaps  involving  the 
destruction  of  the  aircraft  or  over  one-million  dollars  in  dama¬ 
ges  and/or  fatalities.  The  period  reviewed  included  the  data  for 
fiscal  years  1987,  1988  and  1989.  Note  US  Government  fiscal 
years  begin  on  first  day  of  October. 
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Here  113  of  the  169  mishaps  involved  some  type  of  aircrew  error. 
Included  were  32  procedural/  110  perceptualmotor ,  and  157  deci¬ 
sional  errors.  These  types  of  errors  were  labeled  "slips", 
"bungles"  and  "mistakes"  respectively  (Diehl,  1989).  Thus,  this 
data  collectively  reveals  the  significance  of  decisionmaking  to 
all  segments  of  aviation. 


Table  1 


Types  of  Aircrew  Errors  in  Major  Accidents 


Category  of  Error 


Kind  of 
Operation 

Procedural 

"Slips" 

Perceptualmotor 

"Bungles" 

Decisional 

"Mistakes" 

General  Aviation 

5% 

44% 

52% 

Airlines 

19% 

25% 

56% 

Military 

11% 

37% 

53% 

Improving  Aircrew  Reliability 

Over  the  years  great  strides  have  been  made  in  improving  the 
mechanical  reliability  of  aircraft  and  systems,  while  various 
means  of  enhancing  human  reliability  have  also  been  undertaken. 
Some  concepts,  designed  to  prevent  aircrew  error,  were  undertaken 
even  before  the  phenomena  was  thoroughly  understood.  As  noted 
above,  oftentimes  a  priori  scientific  proof  was  unavailable.  A 
proverbial  "Catch  22"  may  have  existed  such  as  "  We  can't  use  it 
until  we  know  it  works,  and  we'll  never  know  it  works  until  we 
try  it."  Other  times  innovations  may  have  just  been  institutiona¬ 
lized  common  sense,  (e.g.  the  development  of  written  checklists 
as  aircraft  became  increasingly  complex).  Such  countermeasures 
have  focused  on  improving  six  human  "faculties"  fundamental  to 
flying . 

These  faculties  constitute  the  "right  stuff"  and  include:  1) 
abilities,  2)  motivations,  3)  knowledge,  4)  procedural 
techniques,  5)  perceptualmotor  skills,  and  6)  decisional 
judgment.  As  I  recently  noted,  Diehl  (1991b),  these  six  items 
are  part  of  a  "Hierarchy  of  Aeronautical  Faculties"  with 
abilities  on  the  base  and  decisional  judgment  at  the  apex,  as  the 
highest  faculty.  Figure  1  depicts  this  relationship  along  with 
the  role  played  by  traditional  and  modern  preventative  measures. 


6 


Obviously,  the  three  categories  of  errors  discussed  above  are 
failures  associated . with  the  three  higher  faculties  (procedures, 
perceptualmotor ,  and  judgment  behaviors).  While  the  other 
faculties  (abilities,  motivations,  and  knowledge)  are  enabling 
factors  which  underlie  such  behavioral  tasks. 

Interestingly,  an  individual  student  aviator  generally  undergoes 
screening  and  training  processes  which  proceed  systematically 
from  the  base  to  the  apex  of  this  hierarchy.  Not  surprisingly, 
this  is  also  the  fundamental  sequence  in  which  the  aviation 
industry  attacked  the  problems  associated  with  human  error. 

Abilities:  Medical  screening  tests  were  successfully  used  in  the 

t’lrst  World  War.  Early  testing  focused  on  easily  measurable 
items  such  as  cardiovascular  health  and  visual  acuity.  Basic 
mental  capacities  were  also  measured.  Such  procedures  were,  of 
course,  improved  over  the  /ears  by  various  military  and  civilian 
organizations. 

Mot ivat ions :  Mental  tests  for  screening  aviation  candidates' 
personalities  and  interests  have  been  widely  used  since  the 
beginning  of  the  Second  World  War.  These  psychological  instru¬ 
ments,  like  the  medical  screening  protocols,  have  been  constantly 
refined.  In  addition,  modern  aviation  organizations  use  a 
variety  of  measures  such  as  employee  assistance  programs  to 
enhance  the  physical  and  mental  faculties  of  their  employees. 

Knowledge:  In  the  post-Second  World  War  era,  greater  emphasis 

was  placed  on  screening  the  prospective  airman's  general  know¬ 
ledge,  (e.g.  a  college  degree  became  a  pre-requisite  for  many 
military  and  commercial  pilot  training  programs).  Imparting  the 
vast  amounts  of  specialized  knowledge  (consisting  of  information 
and  data  as  well  as  rules,  concepts  and  principles)  has  been  a 
major  function  of  aviation  training. 

Great  strides  have  been  made  in  effectively  imparting  knowledge. 
For  instance,  standardized  formats  in  flight  manuals  were  in  wide 
use  by  the  1950s.  Instructional  Systems  Design  concepts  have  been 
used  since  the  1960s  to  systematically  identify  "need-to-know" 
versus  "nice-to-khow"  information.  In  recent  years,  computer 
based  training  applications  have  increased  the  efficiency  of 
teaching  aeronautical  knowledge.  This  knowledge  was  always 
regarded  as  a  prerequisite  for  learning  procedures,  skills,  and 
judgment  tasks. 
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Hierarchy  of  Aeronautical  Faculties 


Figure  1 
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Procedural  Techniques:  These  "finger  faculties"  were  necessary 
to  manipulate  the  switches,  buttons,  and  knobs  of  aircraft  subsys 
terns.  As  on-board  equipment  became  more  complex,  these  techni¬ 
ques  took  on  expanded  importance.  By  the  1950s,  cockpit  design 
standards  and  crew  checklists  were  in  widespread  use.  The 
decades  which  followed  saw  increasing  use  of  devices  such  as  cock 
pit  procedures  trainers  to  enhance  the  mastery  of  such  tasks. 

Perceptualmotor  Skills;  The  importance  of  these  "stick-and- 
rudder"  tasks  have  always  been  recognized.  Good  hand-eye 
coordination  was  a  prerequisite  for  the  timely  maneuvering  of  an 
aircraft  through  three-dimensional  space.  Although,  automation 
and  stability  augmentation  systems  have  decreased  somewhat  the 
amount  of  time  crewmembers  now  spend  on  basic  aircraft  control 
activities.  Control-display  integration  and  fused  sansors  have 
further  decreased  this  type  of  workload.  The  use  of  modern 
digital  simulators  has  facilitated  the  efficient  acquisition  of 
these  skills,  especially  since  the  1960s. 

Decisional  Judgment:  "Headwork"  or  cognitive  tasks  were  also 
regarded  as  vital.  But  such  abilities  were  historically  assumed 
to  be  a  by-product  of  flying  experience,  or  taught  only  informal¬ 
ly.  Little  was  known  about  this  faculty  until  the  1970s.  In  that 
decade,  cockpit  voice  and  flight  data  recorders,  as  well  as  more 
systematic  accident  investigation  methods,  had  revealed  the  magni 
tude  of  judgment,  crew  management  and  situational  awareness  prob¬ 
lems.  Moreover,  human  factors  research  began  suggesting  the 
possibility  of  formally  teaching  cockpit  management  tasks. 

In  that  decade,  the  Federal  Aviation  Administration  ( FAA )  had 
initiated  the  aforementioned  study  into  potential  methods  of 
teaching  judgment  concepts  to  general  aviation  pilots  (Jensen  and 
Benel,  1977).  At  this  time,  USAF  was  examining  methods  to 
improve  the  workload  management  techniques  of  fighter  pilots 
during  emergencies  (Thrope,  Martin,  Edwards  and  Eddowes,  1976). 
Meanwhile,  the  National  Aeronautics  and  Space  Administration 
(NASA)  had  simultaneously  undertaken  several  comprehensive 
programs  focusing  on  methods  of  reducing  crew  errors  in  transport 
aircraft  (e.g.  Ruffle-Smith,  1979).  Several  airlines  were  then 
in  the  process  of  developing  important  training  innovations.  Line 
Orientated  Flight  Training  (LOFT)  was  pioneered  by  Northwest 
Airlines  to  improve  crew  coordination  in  simulators.  United 
Airlines  had  also  initiated  simulator  research  into  subtle 
incapacitation  recognition,  while  KLM  was  developing  a  course  to 
teach  leadership  skills  of  their  line  captains.  The  latter 
program  was,  of  course,  undertaken  as  a  result  of  their  1977 
Tenerife  accident. 
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Implementing  Cockpit  Management  Training 

This  then  was  the  state  of  affairs  in  our  industry  when  a  United 
Airlines  DC-8  crashed  into  a  suburb  of  Portland  Oregon  on  the 
evening  of  Dec  28,  1978.  I  was  dispatched  to  this,  now  well- 
known,  accident  as  the  NTSB  Human  Factors  Group  Co-chairman.  The 
circumstances  of  this  mishap  were  quickly  established:  The  highly 
experienced  crew  became  distracted  by  a  landing  gear  problem  and 
ran  out  of  fuel.  After  reviewing  the  reports  of  similar  accid¬ 
ents  and  the  existing  research,  I  drafted  the  first  recommenda¬ 
tion  calling  for  the  operational  implementation  of  cockpit 
resource  management  programs  by  US  airlines  (NTSB,  1979). 

"Selling"  this  recommendation  to  the  leadership  of  the  NTSB  was 
not  difficult.  For  they  were  easily  persuaded  that  CRM  was  "an 
idea  whose  time  had  come."  These  programs  have  gone  into 
widespread  use  in  the  last  dozen  years.  But,  not  without  much 
debate  about  their  effectiveness,  which  continues  to  this  day. 

Since  writing  that  first  CRM  recommendation,  I  have  found  myself 
immersed  in  the  continued  advocacy,  development,  and  evaluation 
of  such  programs.  These  endeavors  have  included  helping  extend 
the  application  of  cockpit  management  concepts  to  general  avi¬ 
ation  and  military  users.  For  example,  my  first  assignment  after 
becoming  the  FAA  Program  Scientist  for  Human  Performance  was  to 
monitor  the  initial  experiments  on  the  effectiveness  of  the 
prototype  judgment  training  courses  (Buch  and  Diehl,  1984).  When 
the  first  airline  accident  occurred  to  a  crew  which  had  received 
CRM  (NTSB,  1983),  the  FAA  asked  me  to  examine  methods  of 
improving  the  certification  of  these  programs  (Jensen,  1987). 

The  USAF  Inspection  and  Safety  Center  (now  redesignated  as  the 
Safety  Agency)  recently  tasked  me  with  analyzing  data  on  the 
effectiveness  of  civil  and  military  cockpit  management  courses. 
The  results  of  that  research  (Diehl,  1991b)  will  be  discussed 
below. 

This  past  year  has  seen  a  accelerating  interest  in  such  training. 
For  instance,  the  NTSB  as  part  of  a  major  general  aviation 
accident  investigation,  has  recently  recommended  that  Aeronauti¬ 
cal  Decisionmaking  training  be  implemented  among  all  categories 
of  pilots  in  the  civil  aviation  community  (NTSB  1991).  Similarly, 
all  the  USAF  major  commands  now  have  adopted  some  type  of  cockpit 
management  training  program  (Diehl,  1991a). 


Components  of  Cockpit  Management  Training 

Much  has  been  written  in  recent  years  about  methods  for  enhancing 
the  collective  decisionmaking  in  multi-place  aircraft  through  the 
use  of  CRM  techniques  (Lauber,  1984,  Nance,  1986,  Alkov,  1988, 
Foushee  and  Helmreich,  1988,  and  Helmreich,  1991).  In  contrast, 
less  information  is  available  on  programs  aimed  at  improving  the 
decisionmaking  abilities  of  individual  pilots  (Buch  and  Diehl, 
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1984).  Note  that  the  term  "aeronautical  decisionmaking"  (ADM) 
has  become  synonymous  with  "judgment  training"  in  recent  years. 
Furthermore,  categorical  distinctions  between  CRM  and  ADM  are 
disappearing  in  that  today  most  comprehensive  versions  of  these 
programs  have  several  common  functional  components  dealing  with: 
attention,  crew,  stress,  mental  attitude,  and  risk  issues. 

*  The  role  which  the  five  components  or  "cockpit  management  tools" 
play  in  the  hierarchy  of  aeronautical  faculties  is  depicted  in 
Figure  1.  These  tools  may,  in  effect,  provide  "synthetic  experi- 

*  ence"  for  neophyte  airman  while  offering  a  structured  system  for 
assisting  the  decisionmaking  of  their  experienced  counterparts. 
These  five  interrelated  concept  areas  furnish  "rules  and  tools" 
to  help  prevent  common  errors.  For  instance: 

1.  Attention  management  issues  include  understanding  how 
distractions  and  "error  chains"  can  be  avoided. 

2.  Crew  management  issues  teach  the  importance  of  proper 
communications,  division  of  responsibilities,  leadership, 
and  teamwork. 

3.  Stress  management  concepts  focus  on  understanding 
the  effects  of  lifestress  events  as  well  as  providing 
in-flight  stress  coping  strategies. 

4.  Attitude  management  concepts  describe  the  methods  of 
recognizing  and  controlling  certain  hazardous  attitudes 
and  behavioral  styles. 

5.  Risk  management  issues  focus  on  the  rational  evaluation 
of  qualitative  and  quantitative  information  related  to 
operational  hazards. 


Cockpit  Resource  Management:  Note  that  Crew  Resource  Management 
is  another  popular  label  for  such  courses.  The  conceptual  basis 
of  these  programs  was  largely  social  psychology  and  management 
theory.  Many  of  these  programs  were  developed  and  refined  with 
the  data  and  expertise  from  NASA  (Lauber,  1984,  and  Foushee  and 
Helmreich,  1988). 

Most  contemporary  CRM  programs  utilize  training  manuals, 
interactive  classroom  lectures,  with  audio-visual  aids  followed 
by  LOFT  sessions  which  are  video  taped  for  critique  purposes.  The 
courses  provide  a  wealth  of  techniques  to  enhance  flight  deck 
communication.  For  example,  avoid  "excessive  professional 
courtesy":  If  the  captain  is  two  dots  low  on  the  glide-slope, 
tell  him  so  in  unequivocal  terms.  Don't  say,  "  You're  a  little 
low.  Sir". 
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United  Airlines  has  arguably  fielded  the  most  widely  used  of 
these  courses,  although  KLM  launched  the  first  such  course  aimed 
at  captains.  Both  airlines  have  successfully  marketed  their  pro¬ 
grams  to  other  aviation  organizations,  and  they  continue  to 
refine  and  enhance  their  respective  programs  (e.g.  Freeman  and 
Simmon,  1990  and  Siemons,  1991).  Other  airlines  (e.g.  Quantas) 
have  also  independently  developed  highly  innovative  programs 
(Beaumont,  1989). 

The  USAF  Military  Airlift  Command  and  the  US  Naval  Safety  Center 
have  pioneered  militarized  CRM  programs  (Alkov,  1988),  labeling 
them  Aircrew  Coordination  Training  (ACT).  The  USAF  Strategic  Air 
Command  has  recently  fielded  a  very  comprehensive  CRM  program 
under  contract  to  Hernandez  Engineering.  This  course  has,  in 
turn,  been  adapted  for  training  USAF  fighter  crewmembers 
stationed  in  Europe. 

The  USAF  Inspection  and  Safety  Center  initiated  the  development 
of  a  prototype  course  focused  on  single-seat  fighters  in  1990.  It 
was  intended  to  improve  intercockpit  as  well  as  intracockpit 
decisionmaking.  This  program  was  a  modified  and  enhanced  version 
of  the  successful  US  Navy  ACT  program  developed  by  CAE-Link. 

Units  of  the  US  Army  have  applied  this  same  course  to  utility  and 
attack  helicopters.  The  USAF  and  USN  training  commands  have  also 
integrated  these  materials  into  their  respective  undergraduate 
pilot  and  navigator  training.  In  1992  the  USAF  T-l  "Jayhawk" 
trainer  will  become  the  first  operational  system  procured  with 
CRM  specified  in  its  design. 

Aeronautical  Decisionmaking;  The  conceptual  basis  of  the  ADM  or 
judgement  training  programs  was  cognitive  psychology,  for  this 
type  of  training  was  aimed  at  the  attitudes  and  behavior  of  the 
individual  pilot.  Most  of  these  programs  originally  focused  on 
students  pilots  (Berlin,  Gruber,  Holms,  Jensen,  Lau,  Mills  and 
O'Kane,  1982),  but  the  concepts  were  later  applied  to  a  advanced 
training  including  commercial,  instrument,  and  helicopter  pilots 
(Diehl  and  Buch,  1986).  The  FAA,  Transport  Canada,  the 
Australian  Aviation  Department,  and  the  USAF  have  sponsored  the 
development  and  evaluation  of  ADM  programs.  These  materials 
typically  consist  of  training  manuals  and  audio-visual  products 
which  explain  fundamental  concepts  related  to  error  causation  and 
prevent  ion . 

The  way  these  materials  work  is  illustrated  in  Figure  2  from  the 
student  pilot  training  manual,  (Diehl,  Hwoschinsky,  Lawton,  and 
Livack,  1987).  This  figure  depicts  the  decisionmaking  process  as 
a  series  of  feedback  loops  in  which  the  pilots  must  manage 
his/her  attention  in  a  timely  manor  and  sequentially  employ  the 
other  cockpit  management  tools  (for  controlling  stress  etc.). 

The  text  describes  how  one  does  these  things.  Interestingly 
while  this  figure  is  somewhat  simplistic  in  comparison  with  flow 
charts  in  more  sophisticated  texts  (e.g.  Reason,  1990),  it  does 
comport  with  ideas  on  pilot  information  processing  offered  by 
experts  like  Lee  (1990). 
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Situational  Awareness:  Other  similar  cockpit  management  training 
programs  have  focused  on  enhancing  attention  and  task  management 
issues.  For  instance,  when  the  USAF  began  replacing  its  two- 
place  F-4  with  the  single-seat  F-15  in  the  1970s,  concerns  were 
raised  about  pilot  workload  in  emergencies.  Situational 
Emergency  Training  was  undertaken  using  cockpit  procedure 
trainers.  Thus,  pilots  could  practice  diagnosing  typical 
emergencies,  while  maintaining  aircraft  control,  rather  than 
relying  on  memorized  "bold  face"  procedures. 

In  the  early  1980s,  the  US  Air  National  Guard  became  concerned 
about  the  ability  of  their  A-7  pilots  to  maintain  proficiency  in 
the  low-altitude  tactics.  Their  Low  Altitude  Training  program 
was  undertaken  to  teach  pilots  techniques  for  overcoming  the 
unique  hazards  of  operating  in  this  highly  dangerous  and  time 
critical  environment  (e.g.  the  tendency  to  fly  lower  over  small 
desert  bushes  because,  at  high  speed,  they  appear  to  be  the  same 
size  as  the  larger  trees  which  one  is  used  to).  This  program 
included  academics,  simulator,  and  flight  training. 

Another  important  situational  awareness  training  effort  was  just 
announced  by  the  USAF  Tactical  Air  Command.  Their  Aircrew 
Attention  Awareness  Management  Program  is  designed  to  acquaint 
fighter  pilots  and  weapon  systems  officers  with  physiological  and 
psychological  factors  affecting  their  performance.  These 
concepts  are  being  taught  in  part  by  specially  trained  physio¬ 
logists  (familiar  with  CRM,  ADM,  etc.)  who  have  been  assigned  to 
each  fighter  training  unit. 


Measuring  Training  Effectiveness 

These  training  programs  have  all  been  generally  well  received  by 
the  individuals  and  organizations  which  have  used  them.  Much 
contemporary  research  has  described  the  improvement'^the  attitudes 
of  people  enrolled  in  such  programs.  But,  for  the  reasons  noted 
earlier,  proving  that  the  programs  have  prevented  errors  or 
reduced  mishaps  rates  is  difficult. 

Fortunately,  there  is  anecdotal  information  that  such  programs 
have  halped  prevent  mishaps  in  a  wide  variety  of  civilian  and 
military  aircraft.  For  instance,  a  US  Navy  A-6  crew  experienced 
a  total  hydraulics  failure,  but  was  able  make  a  safe  landing. 

The  investigation  concluded  that  this  was  a  "first-ever"  in  that 
type  aircraft,  and  their  aircrew  coordination  training  was  a 
factor  in  this  "save"  (Alkov,  1991b).  The  NTSB  came  to  basically 
the  same  conclusion  regarding  the  value  of  CRM  training  in  a 
similar  incident  involving  the  United  Airlines  DC-10  at  Sioux 
City,  Iowa  (NTSB,  1990). 
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Preventing  Error:  One  of  the  best  methods  of  examining  the 
effectiveness  of  training  programs  is  to  perform  empirical  tests 
to  document  whether  crewmembers  who  receive  such  training  make 
fewer  errors.  ADM  programs  have  been  extensively  tested  in  this 
way.  That  was  partly  because,  unlike  CRM  which  was  primarily 
employed  for  airline  and  military  operations,  ADM  was  initially 
applied  to  general  aviation  student  training  situations.  The 
latter  environment  obviously  involves  relatively  high  error  rates 
and  low  costs,  thus  permitting  the  use  of  controlled  experiments. 

Worldwide  there  have  been  six  government  sponsored,  independent, 
evaluations  of  the  ADM  training  programs.  A  detailed  description 
of  this  research  was  recently  completed  (Diehl,  1990).  These 
evaluations  were  performed  to  ascertain  the  effectiveness  of  such 
materials  under  differing  conditions.  The  basic  criteria  were 
errors  made  during  short,  seemingly  routine,  crosscountry  "obser¬ 
vation  flights."  On  these  flights,  specially  trained  observers 
surreptitiously  placed  subjects  in  a  series  of  specific  decision¬ 
making  situations  (e.g.  rushing  preflight  inspections,  or 
suggesting  steep  maneuvers  at  low  altitudes).  Observers  then 
unobtrusively  recorded  the  errors  on  these  judgment  items.  In 
these  rigorous  "double-blind"  experiments,  the  observers  were  not 
informed  which  subjects  had  received  ADM  training,  while  subjects 
were  unaware  of  the  real  purpose  of  the  flights  beforehand  (e.g. 
subjects  might  be  lead  to  believe  they  would  be  evaluating  new 
map  designs). 

As  expected,  the  effectiveness  of  the  ADM  materials  varied  widely 
depending  primarily  upon  the  comprehensiveness  of  the  training 
(see  Table  2).  For  the  six  studies,  the  improvement  ranged  from 
8%  in  a  voluntary,  minimally  structured,  situation  to  46%  for  a 
well  structured,  comprehensive,  ground  school  environment  with 
simulator  training.  Note  that  all  six  tests  were  statistically 
significant  at  or  beyond  the  .05  level  of  confidence. 
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Table  2 


ADM  Training  Experimental  Evaluations 


Sponsor/Sub jects 

Methods 

Results/Researchers 

Australian  Gov't. 
Private  Pilots 

Manuals,  Lectures, 
Flight  Training 

8%  fewer  errors 
(Telfer  and  Ashman, 
1986) 

Transport  Canada 
Private  Pilots 

Manuals 

only 

9%  fewer  errors 
(Buch  and  Diehl, 
1983) 

US  FAA 

Private  Pilots 

Manuals 

only 

10%  fewer  errors 
(Diehl  and  Lester, 
1987) 

US  FAA 

Student  Pilots 

Manuals,  Lectures, 
Flight  Training 

17%  fewer  errors 
(Berlin,  et  al., 
1982) 

Transport  Canada 
Civilian  Cadets 

Manuals,  Lectures, 
Flight  Training 

40%  fewer  errors 
(Buch  and  Diehl, 
1982) 

USAF 

Instrument  Students 

Manuals,  Lectures, 
Simulator  Training 

46%  fewer  errors 
(Connolly  and 

Blackwell,  1987) 


Preventing  Accidents 

These  experimental  evaluations  provide  strong  statistical  evi¬ 
dence  such  training  can  change  behavior,  and  thereby  reduce 
errors  in  low-time  general  aviation  pilots.  But  the  fundamental 
criteria  for  evaluating  the  effectiveness  of  cockpit  management 
training  programs  is  their  ability  to  reduce  the  accident  rates 
in  the  broader  "operational  world." 

Airlines  are  obviously  the  most  numerous  users  of  these  programs. 
But,  their  accidents  occur  very  infrequently,  (perhaps  once  every 
few  years  for  a  particular  operator).  Thus,  it  would  be  very 
difficult  to  prove  that  an  individual  airline,  which  had  adopted 
CRM,  in  fact  had  experienced  a  significant  decline  in  their 
aircrew  error  accident  rate. 
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Fortunately,  the  FAA  and  Transport  Canada  have  developed  versions 
of  the  ADM  training  manuals  for  helicopter  pilots  (Adams  and 
Thompson,  1987).  This  manual  became  widely  used  by  a  number  of 
major  rotorcraft  organizations  (Adams  and  Diehl,  1988).  Because 
the  accident  rates  for  rotorcraft  (and  military  aircraft)  are 
normally  orders  of  magnitude  higher  than  those  for  airliners,  one 
should  be  able  to  more  easily  detect  improvement  in  their 
records.  This  was,  in  deed,  the  case. 

Bell  Helicopters  Textron  Inc.  ( BHTI ) :  This  major  rotorcraft 
manufacturer  provides  extensive  initial  and  recurrent  training  in 
both  the  US  and  abroad.  They  have  utilized  the  ADM  materials 
(Adams  and  Thompson,  1987)  in  their  training  programs  since  they 
were  first  published  (Fox,  1991).  BHTI  particularly  targeted 
their  popular  Bell  Model  206  "Jetranger"  because  that  craft  gen¬ 
erated  about  46%  of  the  total  US  civil  helicopter  flying  hours. 

They  have  also  developed  a  "Cockpit  Emergency  Procedures  Expert 
Trainer."  Fox  (1991)  described  this  system  as  an  artificial 
intelligence  based  software  package  which  allows  a  pilot  to  use  a 
personal  computer  as  a  decisionmaking  simulator. 

The  results  of  BTHI  ADM  training  efforts  are  impressive, 
especially  when  their  accident  rates  are  examined.  Fox  (1991) 
compared  the  1983-1986  period  (before  training  was  begun)  with 
the  1987-1990  period.  The  world-wide  human  error  accident  rate 
(per  100,000  hours)  declined  by  36%  for  the  Jetranger.  Note  for 
comparison  purposes,  the  rate  for  mechanically  caused  accidents 
declined  by  only  8%. 

Fox  (1991)  also  stated  the  US  Jetranger  human  error  accident 
rates  declined  by  an  even  more  impressive  48%.  Here  the  compari¬ 
son  periods  were  1984-1986  and  1987-1988.  He  notes  that  many 
Jetranger  pilots  attending  their  training  also  fly  other  single¬ 
engine  helicopters,  which  may  partly  explain  the  more  modest  25% 
improvement  in  those  rates  during  this  period. 

Petroleum  Helicopter  Inc.,  (PHI):  This  organization  is  the 
largest  commercial  helicopter  operator  in  the  US  with  approxi¬ 
mately  300  helicopters  and  fixed-wing  aircraft.  The  company 
historically  has  had  an  excellent  accident  rate,  well  below  the 
industry  average.  Their  chief  pilot  (Mr  Vern  Albert)  reported 
the  results  of  the  using  of  ADM/CRM  training  in  Rotor  &  Wing 
International  (1989),  p.  65:  "From  1980  through  1986,  we  had  an 
accident  rate  of  about  2.3  accidents  per  100,000  flight  hours.  In 
mid-1986,  we  started  ADM  training,  and  the  rate  in  1987  was  1.86 
and  then  dropped  to  1.05  in  1988.  The  only  thing  we  changed  in 
our  training  syllabus  was  adding  ADM  and  cockpit  resources 
management."  This  translated  to  a  54%  reduction  in  their  overall 
accident  rate. 
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US  Navy;  In  1986,  the  Naval  Safety  Center  reviewed  the  CRM  pro¬ 
grams  which  were  underway  at  several  airlines  and  the  USAP  Mili¬ 
tary  Airlift  Command  (Alkov,  1988).  They  began  formal  CRM  train¬ 
ing  at  all  Navy  and  Marine  Corps  helicopter  training  units  in 
1987.  CRM  was  then  initiated  in  their  A-6/EA-6  "Intruder" 
fighter-bomber  training  units  in  1988.  As  noted  earlier  they 
labeled  these  CRM  materials  Aircrew  Coordination  Training. 

Alkov  (1991a),  p.  25,  stated  "Aircrew  error  mishaps  rates  for 
helicopter  and  the  A-6/EA-6  communities  have  declined  dramati¬ 
cally  since  the  introduction  of  Aircrew  Coordination  Training." 
Comparing  the  data  for  the  fiscal  year  before  the  CRM  training 
began  with  the  most  recent  fiscal  year  data  supports  this  state¬ 
ment.  For  these  fighter-bombers,  their  1990  aircrew  error  rate 
for  all  mishaps  was  of  1.43.  Compared  with  their  1986  rate  of 
7.56,  this  represents  an  81%  improvement.  Similarly,  for  their 
helicopters,  the  1990  rate  of  5.05  versus  the  1986  rate  of  7.01 
represents  a  28%  improvement.  Incidentally,  their  tentative 
figures  for  1991  suggest  that  both  helicopter  and  A-6  mishap 
rates  continued  to  decline  by  another  several  percent  (Aihov, 
1991b). 

USAP  Military  Airlift  Command:  In  1985,  MAC  became  the  first 
military  organization  to  adopt  CRM  training.  This  program  was 
initiated  by  the  MAC  Commander  (General  Thomas  Ryan,  Jr.)  and 
labeled  Aircrew  Coordination  Training.  MAC  has  several  thousand 
aircrewmembers  who  operate  almost  1,000  transports  and 
helicopters  and  fly  approximately  700,000  hours  annually.  Thus 
it  undoubtedly  qualifies  as  one  of  the  larger  organizations  to 
embrace  CRM  concepts.  The  CRM  materials  were  developed  by 
individual  MAC  units  and  several  contractors  including:  United 
Training  Systems,  Flight  Safety  International,  Hughes,  and 
CAE-Link. 

The  MAC  safety  record  for  the  five  fiscal  years  before  CRM 
(1981-1985)  was  compared  with  the  five  years  (1986-1990)  after 
they  adopted  this  training.  The  total  number  of  aircraft  destro¬ 
yed  dropped  from  21  to  10  (a  52%  improvement).  Similarly  their 
Class  A  and  B  operat ions-related  flight  mishap  rate  dropped  by 
51%  (from  .679  to  .333  per  100,000  hours).  Note  that  these 
events  currently  include  mishaps  involving  aircrew  errors  which 
the  damages  exceeded  $200,000.  These  improvements  far  outpaced 
the  rest  of  the  USAF  which  saw  the  number  of  aircraft  destroyed 
decrease  by  18%  while  their  aircrew  error  mishaps  dropped  by  21%. 

This  ten  year  period  was  relatively  stable  operationally, 
although  longitudinal  comparisons  can  be  fraught  with  dangers. 
There  obviously  were  a  number  of  other  training  related 
developments  occurring  at  this  time  (such  as  the  simulator 
upgrades  made  throughout  the  USAF).  However,  the  one  major 
difference  between  MAC  training  and  that  of  the  other  commands 
during  this  period  was  the  use  of  CRM.  It  is  also  possible  that 
the  MAC  Commander's  bold  decision  to  undertake  this  type  training 
for  all  aircrewmembers  produced  other  desirable  side-effects. 
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The  two  earlier  USAF  situational  awareness  programs  have  also 
been  found  to  work  well.  For  instance.  Situational  Emergency 
Training  was  judged  more  effective  than  conventional  training  it 
replaced  (Thorpe,  et.al.,  1976),  while  the  manager  of  the  Low 
Altitude  Training  program  stated  that  of  the  approximately  400 
graduates  from  their  course,  only  one  had  been  involved  in  a 
collision  with  the  ground  mishap  (Thomae,  1991). 


Table  3 


ADM/CRM  Operational  Evaluations 


Organization/Subjects 

Materials 

Accident  Rates 

Bell  Helicopters  Inc. 

Worldwide,  Jetranger  Pilots 

ADM 

36% 

Decrease 

Bell  Helicopters  Inc. 

US  Only,  Jetranger  Pilots 

ADM 

48% 

Decrease 

Petroleum  Helicopters  Inc. 
Commercial  Pilots 

ADM  &  CRM 

54% 

Decrease 

US  Navy 

All  Helicopters,  Crewmembers 

CRM 

28% 

Decrease 

US  Navy 

A— 6  Intruder,  Crewmembers 

CRM 

81% 

Decrease 

US  Air  Force 

MAC  Transports,  Crewmembers 

CRM 

51% 

Decrease 

Discussion 

Table  3  summarizes  the  evidence  reported  to  date  on  the  effects 
of  using  ADM  and  CRM  training  by  several  large  organizations.  In 
all  six  instances,  the  use  of  the  training  was  followed  by  major 
reductions  in  their  accident  rates.  Collectively,  these 
improvements  were  statistically  significant  at  beyond  the  0.02 
level  of  confidence.  These  results  agree  with  the  six  controlled 
experiments  which  focused  on  low-time  general  aviation  pilots. 
Furthermore,  this  operational  user  data  covers  a  large  variety  of 
civil  and  military  flying  from  light  helicopters,  and  medium 
fighter-bombers  to  heavy  transports.  It  is,  of  course, 
impossible  to  conclude  that  CRM/ADM  training  was  solely 
responsible  for  all  the  improvements  observed.  However,  the 
inclusion  of  such  training  by  management  may  act  as  a  catalyst 
for  other  beneficial  behaviors  which,  in  turn,  can  reduce 
mishaps. 
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Povenmire  et  al.  (1989),  in  their  classic  simulator  study,  noted 
that  those  crews  who  "innately"  know  how  to  use  CRM-like  tech¬ 
niques  are  more  effective.  Similarly  Helmreich  (1991)  shows 
those  who  have  had  CRM  training  outperform  their  untrained  coun¬ 
terparts.  These  innovative  concepts  have  been  endorsed  by  a  host 
of  prestigious  organizations  (e.g.  Flight  Safety  Foundation, 
Aerospace  Medical  Association,  Air  Line  Pilots  Association). 
Furthermore,  the  International  Civil  Aviation  Organization  has 
recommended  their  use  for  training  all  newly  licensed  pilots, 
(ICAO,  1989).  Ambitious  research  and  development  efforts  are 
also  underway  at  various  organizations  such  as  the  US  Naval 
Training  Systems  Center  (Hartel,  Smith,  and  Prince,  1991)  and  the 
FAA  and  NASA  (Kayten  and  Fou3hee,  1991).  This  work  will 
undoubted  further  enhance  these  programs. 

These  concepts  may  also  have  applications  beyond  flightcrew 
training,  witness  the  fact  that  many  airlines  now  include  such 
materials  in  flight  attendant  courses.  The  FAA  has  also 
published  decisionmaking  materials  for  air  ambulance  program 
administrators  (Rotor  &  Wing  International,  1989),  while 
Continental  Airlines  has  applied  the  concepts  to  maintenance 
personnel,  (Fotos,  1991).  Not  surprisingly,  the  US  Army  provides 
such  training  to  their  safety  officers  (Lofaro,  1990),  while  the 
US  Pacific  Air  Forces  now  includes  this  training  for  new  squadron 
commanders  (Diehl,  1991a).  Lastly,  Kayten  and  Foushee  (1991) 
correctly  note  the  similarities  between  CRM  and  industrial  total 
quality  management  programs. 

New  versions  of  this  training  have  been  labeled  "third  generation 
CRM  programs"  (Johnson,  Shroyer,  and  Grew,  1991).  In  fact,  I 
believe  the  more  sophisticated  programs  have  several  character¬ 
istics:  1)  focusing  on  enhanced  effectiveness  as  well  as  safety, 

2)  targeting  broader  audiences  (e.g.  instructors,  flight 
attendants,  etc.),  3)  having  built-in  update  mechanisms,  4)  using 
computer-based  training,  and  5)  employing  humor  and  aesthetics. 

In  the  Air  Line  Pilots  Association  1991  annual  workshop  it  was 
interesting  to  hear  candid  discussions  of  still  unresolved  issues 
associated  with  cockpit  management.  It  was  reminiscent  of  my 
earlier  concerns  about  how  to  evaluate  or  certify  the  people 
enrolled  in  such  programs,  as  well  as  how  to  rehabilitate  those 
crewmembers  who  reject  or  "fail"  this  training,  or  who  have 
exhibited  poor  judgment  in  an  incident,  accident  or  violation 
(Diehl,  1982).  I  still  feel  that  such  issues  may  be  more 
recalcitrant  than  our  basic  concerns  about  cockpit  management 
education  and  training. 
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Conclusions 


The  results  of  these  six  empirical  and  six  operational 
evaluations  provide  strong  evidence  that  these  training  programs 
can  help  reduce  aircrew  errors  and  thereby  prevent  accidents. 
Furthermore,  categorical  distinctions  between  CRM  and  ADM 
training  are  becoming  blurred  in  that  most  current  versions  of 
these  programs  have  five  common  elements,  which  provide  tbols  to 
deal  with  attention,  crew,  stress,  attitude  and  risk  management 
issues.  While  additional  research  and  development  conti  ,ues  there 
is  a  growing  realization  that  these  programs  ideally  need  to  be 
introduced  early  in  flight  training,  reinforced  during  upgrade 
training,  and  reviewed  during  recurrent  training. 
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and  the  FAA.  He  holds  academic  degrees  in  psychology,  management 
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Instructors  licenses.  The  views  expressed  in  this  article  are 
those  of  the  author  and  do  not  necessarily  reflect  the  official 
policy  or  position  of  the  Department  of  Defense  or  the  US 
Government. 
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COCKPIT  DECISION  MAKING 


Conclusion: 

Mishap  rales  associated  with  aircrew  errors  decreased 


from  28%  to  81%.  There  is  also  evidence  that  such  training  can 
enhance  effectiveness  as  well  as  safety. 
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imlrprmlrnilv  ilnrlnpnl  -imilar  prnnram-. 

I  hr  l  S.  Air  I  nrrr  Militarv  Mrlilt  <  nnmiaml  (MA(.) 
ami  ihr  Naval  Salrn  kentrr  liavr  pionrrrrd  militari/rd 
(  HM  prnnram-.  lahrling  (linn  \inrrw  ( noplinalinn 
I  raining  (A(  I  ).  Ihr  Sirairgir  Mr  (ommand  (S\(!) 
lalrr  ailoptnl  -m  il  training.  Brran-r  nl  (hr  -irnng  nrnl 
Inr  ihi-  ivpr  nl  iraininn  in  tartiral  airrralt  anil 
hrlirnpirr-.  thr  I  S.  Air  I  nrrr.  \rim  anil  Navv  arr  now 
n-iii"  -nrh  rnnr-r-  In  rnhanrr  teamwork  in  iillrrrorkpil 
a-  well  a-  mlrarnrk|MI  la-k- 

(IBM  prnnram-  nnrralk  n-r  iraininn  manual-  ami 
mlrrartivr  rla— rnnm  Irriurr-  with  amlin-u-iial  aid-, 
followed  liv  prarlirr  -r— inn-  ill  -imiilalnr-  Ideallv. 
\ iilrn  lapr-  arr  Iliailr  In  provide  Irnlhark  In  trailin'-  on 
milivnlu.il  ami  rrrw  prrfnrmanrr  during  I  .Of  l/MOSI 
fliizhl  -imiilalnr  iraininn  -r— inn-  \ firr  ihr  -r— inn-  arr 
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rrifi<|ucd.  I  hr  tapes  arc  normally  c  rax'd  to  ensure  future 
spontaneity,  candor,  and  realism.  The  FAA  issued  an 
Advisory  (areolar  on  CRM  in  TIRO. 

Aeronautical  Decision  Making 

The  second  general  category  of  programs  was  formerly 
called  "Judgment  Training."  The  conceptual  basis  of 
Aeronautical  Decision  Making  (ADM)  programs  was 
cognitive  psychology,  for  this  type  of  training  is  aimed 
at  the  attitudes  and  behavior  of  the  individual  pilot. 
Most  of  these  programs  originally  focused  on  student 
pilots,  hut  were  later  applied  to  advanced  training 
including  commercial,  instrument,  and  helicopter 
pilots.  The  T  AA.  Transport  Canada,  the  Australian 
Aviation  Department,  and  the  Air  Force  have  sponsored 
the  development  and  evaluation  of  ADM  programs. 
These  materials  typically  consist  of  training  manuals 
and  audio-visual  materials  which  explain  fundamental 
concepts  related  to  error  causation  and  prevention. 

The  T  AA  has  been  involved  in  CRM/ ADM  program 
development  anil  research  for  a  number  of  years.  In 
t(T87.  the  I  AA  began  publishing  manuals  oriented  to  the 
decision  making  needs  of  various  categories  of  pilots. 
These  manuals  provide  information  such  as  attitude 
modification,  risk  recognition,  and  stress  management. 

Common  functional  elements 

The  categorical  distinctions  between  CRM  and  ADM  are 
disappearing.  Today,  most  eomprrhensivr  versions  of 
these  programs  have  several  common  functional, 
interrelated  elements  (Figure  I)  dealing  with  attention, 
crew,  stress,  mental  altitude,  and  risk  issues.1 2  The  role 
which  these  five  "cockpit  management  tools”  play  in 
the  hierarchy  of  aeronautical  faculties  is  depicted  in 
Figure  1  They  may  in  effect  help  provide  "synthetic 
experience.”  Those  five  interrelated  concept  areas 
furnish  "rules  and  tools”  to  help  prevent  common 
errors.  Tor  instance: 

1.  Attention  management  issues  include  understanding 
how  distractions  and  "error  chains”  can  be  avoided. 

2.  Crew  management  issues  teach  the  importance  of 
proper  communications,  division  of  responsibilities, 
leadership,  and  teamwork. 

,'F  Stress  management  concepts  focus  on  understanding 
the  effects  of  life  stress  events,  as  well  as  providing 
in-fliglu  'tress  coping  strategies. 

A.  Attitude  management  concepts  describe  the  methods 
of  recognizing  and  controlling  certain  hazardous 
attitudes  and  behavioral  sty  les. 

Risk  management  issues  focus  on  the  rational 
evaluation  of  i|ualitative  and  ipiautilative  informa¬ 
tion  related  to  operational  hazards. 


Training  effectiveness 

CRM  ami  ADM  programs  have  been  generally  well 
received  bv  the  individuals  and  organizations  that  have 
used  them.  Furthermore.  NASA  and  T  AA  research  has 
documented  changes  in  the  attitudes  of  crewmembers. 

There  is  also  anecdotal  information  that  programs 
have  helped  prevent  certain  mishaps.  For  instance,  in 
1  'Hi').  a  Tinted  Airlines  DC- 10  suffered  an  uneoulained 
failure  of  the  center  engine  anil  the  total  loss  of  flight 
control  effectiveness.  The  crew  manipulated  the 
outboard  engines  to  maintain  enough  control  to  allow 
an  emergency  landing  at  Sioux  City.  Iowa,  in  which 
most  of  the  occupants  survived.  The  National  Traffic 
Safety  Board  concluded  that  CRM  training  was  valuable 
in  preventing  what  might  have  been  an  even  more 
catastrophic  accident. 

Another  method  of  examining  the  effectiveness  of 
training  programs  is  to  perform  controlled  experiments 
to  document  whether  crewmembers  who  receive  such 
training  make  fewer  errors.  ADM  programs  have  been 
extensively  tested  in  this  way.  That  was  partly  because, 
unlike  CRM  which  was  primarily  employed  for  airline 
and  military  operations.  ADM  was  initially  applied  to 
general  aviation  student  training  situations.  The  latter 
environment  involves  relatively  high  error  rales  and  low 
costs,  thus  facilitating  the  use  of  experimental  studies. 

A  recent  paper  summarizes  the  evidence  regarding 
the  effectiveness  of  such  training.  '  In  several  controlled 
experiments,  inexperienced  pilots  who  were  given  judg- 


Table  1 

Experimental  Evaluations 

Sponsor 

Subjects/ 

Environments 

Improvements 

Australian 

Aviation  Dept. 

Private  Pilots/ 

Light  Airplanes 

8%  Error  Decrease 

Transport  Canada 

Private  Pilots/ 

Light  Airplanes 

9%  Error  Decrease 

U.S.  FAA 

Private  Pilots/ 

Light  Airplanes 

10%  Error  Decrease 

U.S.  FAA 

Student  Pilots/ 

Light  Airplanes 

17%  Error  Decrease 

Transport  Canada 

Civilian  Cadets/ 

Light  Airplanes 

40%  Error  Decrease 

USAF 

Instrument  Students/ 
Simulators 

46%  Error  Decrease 

Fiye 

Characteristics 

of  New 

(Comprehensive) 

Programs 

1.  Focus  on  enhanced 
effectiveness  as 
well  as  safety 

2.  Target  broader 
audiences  (e.g., 
instructors,  flight 
attendants,  etc.) 

3.  Have  built-in 
update 
mechanisms 

4.  Utilize  computer- 
based  training 

5.  Employ  humor  and 
aesthetics 


'l)i.  I.l  y  I  [  In-  I  ffiiinriir.,  I  r.  lining  l’r«.i!,;Hii-  ft»r  y  iri  r*  m  I  1 1  ,»r  In  tVH-rc.lii,".  "I  llir  s,\ll,  Iiim-uijIiimmI  s,„,|m,.iiihi  nf  X\i;itmn 

P»trlMtl<ijn  ( Hiin  Sat*' I  I  iilmolur-  Olim  \j»ril  J1*  •  \|m  2  1‘^M 

-n, i.i 
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Table  2 

ADM/CRM  Operational  Evaluations 


Organization/Subjects 

Materials 

Accident  Rates 

Petroleum  Helicopter,  Inc/ 

PHI  Commercial  Pilots 

ADM  &  CRM 

54%  Decrease 

Bell  Helicopter/ 

Worldwide  Jetranger  Pilots 

ADM 

36%  Decrease 

Bell  Helicopter/ 

U.S.  Jetranger  Pilots 

ADM 

48%  Decrease 

U.S.  Navy / 

Helicopter  Crewmembers 

CRM 

28%  Decrease 

U.S.  Navy/ 

A-6  Intruder  Crewmembers 

CRM 

81%  Decrease 

U.S.  Air  Force/ 

MAC  Transport  Crewmembers 

CRM 

51%  Decrease 

program^  ideallv  ihtiI  in  l>r  introduced  curb  in  fliiihl 
training.  reinforced  (luring  upgrade  training.  uml 
reviewed  during  recurrent  training  uml  accident 
|irt*\ a'lllion  'eminur*  ^ 

hr  Dr.  I  Inn  /..  Diehl. 

Dr.  Dielil  is  the  Technical  1  tlnsor  for  Human 
Performance  at  the  I  ..V  Mr  Force  Safety  Agency, 
\orton  lir  Farce  Hose.  Californio,  lie  has  over  20- 
vears  experience  with  aircraft  manufacturers,  the 
I  uiteil  States  \ary.  \alioniil  Traffic  Sajcly  llaaril. 
mul  FA  I.  lie  holds  academir  degrees  in  psychology 
management .  and  engineering,  as  well  as  Airline 
Transport  Pilot  anil  Certified  Instructor  licenses.  Dr. 
Diehl  drafted  the  first  I  ,S.  Corerument  recom¬ 
mendation  calling  for  the  implementation  of  CRM 
programs  and  has  been  in  ml  red  with  the  decelopmenl 
of  CRM  and  ADM  concepts  for  eiril  and  military 
applications.  The  news  expressed  herein  are  the 
author's  and  do  not  necessarily  represent  those  of  the 
I  S  I F  or  any  other  gocernmental  agency. 

This  sigmtnt  ot  at  Journal  promts  Ot  trillion  community 
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im-iil  training  inuilf  8  in  4(»  (irri  fnl  fewer  errors  limn 
those  who  rn  fi\fil  i-nmniiiniiul  curricula  ('ft-  I'nlilr  I ). 
More  iiii|>ortaiilly.  niher  data  has  confirmed  that  inishap 
rnifs  associated  with  aircrew  errors  decreased  dminuti- 
cally  in  organizations  i  hat  instiiuifil  ADM  nr  CRM 
training  programs.  Tin*  amoimi  of  this  iui|iri)Vfinfiil 
ranged  from  28  in  81  itercciii  (set  Table  2).  For  exam- 
pie.  in  iht*  5  years  after  MAC  iiisiiiuieil  this  training, 
their  rates  improved  by  51  (tercettl.  while  those  for  the 
rest  of  the  l  SAF  improved  only  21  (terernl.  There  is  ulsn 
evidence  indicating  that  such  (echniiptes  can  enhance 
effectiveness  as  well  as  safety  (e.g.  B-52  liomb  accitmcv). 

Conclusions 

The  categorical  distinctions 


mta  an  opportunity  to  put  list  ptptrs  ot  signttkant  trillion 
stilly  topics.  Wo  milcomi  proltssiontls  it  the  Hold  ol 
iriitiot  a  submit  ptptrs  tor  publication.  Wo  encourage  you 
to  supplement  Ot  papers  with  starts,  graphs,  IHnstraUons  ami 
photographs.  Proposed  papers  should  be  to  totper  Oat  3,119 
words.  All  papers  become  the  property  ol  too  Federal  Mutton 
Administration  (FAA).  Tho  FAA  Aria  Hot  Safety  Journal  staff 
reserves  the  right  to  choose  which  papers  are  printed  and  to 
make  editorial  modifications  without  changing  the  meaning. 
Please  sand  yonr  paper  to: 

FAA  Aviation  Satoty  Journal 
Safety  Information  Stall.  ASF-29 
U.S.  0  apartment  01  Transportation 
800  Indapandinco  Avenue  S.W. 

Washington,  0C  20591 


between  CRM  and  ADM 
training  are  becoming  blurred. 
Most  current  versions  of  these 
programs  have  five  common 
elements  which  deal  with 
attention,  crew,  stress,  attitude, 
and  risk  management  issues. 
The  results  of  six  empirical  and 
six  operational  evaluations 
(Tables  1  and  2)  provide  strong 
evidence  that  these  training 
programs  can  help  reduce 
aircrew  errors  and  therein 
prevent  accidents.  While 
additional  research  and 
development  continues,  there  is 
a  growing  realization  that  these 
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Captain  Stephan  1.  Fan! 


Delta  Air  Lines 


DELTA  AIR  ONES 
DECISION  MAKING  TRAINING 


Conclusion: 

Initially,  CRM  and  ADM  training  was  too  academic  and  the 
operational  impact  was  lost  in  "psychobable*.  Emphasis  on 
operationally  oriented  training  (i.e.,  experience  based)  creates 
the  value  or  benefit  first,  which  facilitates  the  pilot  attitude 
changes  required  to  change  behavior. 
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DELTA  A  I  Ft  LINES 
DECISION  MAKING  TRAINING 

I.  Introduction 

A.  Initial  CRM  Training  -  Decision  Dynamics  -  2:45 

B.  Transition  -  Problems,  Solutions 

C.  '92  Recurrent  Training  -  2:00 
II.  Decision  Dynamics 

A.  Sample  decision  problem 

B.  P.A.S.S.  Model 

C.  Heuristics 

1 .  Sv  r  tcut 

2.  Jhort  circuit 

3.  Types;  operational  examples;  short  circuit  P.A.S.S. 

a.  Representativeness 

b.  Availability 

c.  Anchoring 

d.  Overconfidence 

4.  Delta  incidents 

D.  Habitual  routines 

1.  Definition 

2.  Disconnects 

E.  Situational  awareness 

1.  Plan 

2.  Communicate 

3.  Critique 

4.  Monitor 

DELTA  AIR  LINES  May  6,  1992 
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F.  Better  to  disprove 
III.  Transition 

A.  Problem:  Academic  vs.  Operational 

1.  Initial  CRM  too  academic 

2.  Operational  examples  lost  in  "psychobabble” 

3.  Sequence  and  set-up  more  than  total  content 

B.  Solution:  Operationally  oriented  training 

1.  Create  value,  get  attitude  &  behavior  shifts 

2.  Hangar  talk  -  build  consensus  meaningful  events 
"that  could  happen  to  me” 

3.  Support  THEIR  conclusions  with  academic  research 
IV.  1992  Recurrent  Training  -  Total  Crew  Awareness 

A.  Incidents 

1.  Operational  orientation  -  hangar  talk 

2.  Recent,  in-house 

B.  Crew  Performance  Categories  -  how  we’re  evaluating 

1.  Systems  Knowledge 

2.  Procedural  Execution 

3.  Control  Technique  and  Execution 

4.  Command  Leadership 

5.  Crew  Coordination 

6.  Management  Skills 

C.  Demonstrated  proficiency  in  Performance  Categories 

1.  Blue  Angel /Thunderbird  video  -  why  good 

2.  Airline  operations  -  same/different 

3.  Being  "predictable" 

DELTA  AIR  LINES  May  6,  1992 
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D.  Total  Crew  Awareness 


1.  Def.  Roles,  Responsibilities,  Expected  Sequence  of 
Events 

2.  Generic  emergency  -  BOLDFACE  -  forced  TCA 

3.  Gear  problem  after  take-off  -  abnormal ,  create  TCA 

4.  Video  -  AA  135 

5.  Routine  arrival  -  plan?  brief?  incident 

E.  Creating  TCA 

1.  Captains!  what  can  F.O./S.O.  do  to  create  TCA? 

2.  F.O./S.O.  what  can  Captain  do  to  create  TCA? 

3.  Anticipate,  Plan,  Communicate 

F.  Supporting  research  -  NASA  Ames;  J.  Orasanu 

1.  NASA  Loft  from  hell 

2.  Characteristics  of  High  Performance  Crews 

a.  Anticipate  difficulties  early 

b.  Use  low  workload  periods  effectively 

c.  Talk  less  when  workload  high 

d.  Seek  more  information  from  others  earlier 

V.  Q  &  A 


DELTA  AIR  LINES 


May  6,  1992 


33 


United  Air  Limes 


DECISION  TASK  ANALYSIS  FOR  THE 
AIR  TRANSPORT  PILOT 


Conclusion: 

Decision  snaking  is  one  of  many  demands  on  the  crew  system. 
ADM  training  should  he  based  on  the  key  decision  points  in  the 
mission  with  special  emphasis  on  those  points  that  are  clues  to 
loss  of  situational  awareness. 
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DECISION  TASK  ANALYSIS 
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Opcon  Corporation 


GENERAL  DECISION  OBJECTIVES  WITH  RESPECT  TO  RISK  PROFILE 
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OPERATIONAL  INTEGRETY 
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DECISION  STRUCTURE 
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Modify  L  Nav  to  accommodate 
terrain/obstruction/weather. 


DECISION  STRUCTURE 
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EXAMPLE:  Flying  HKG  ELATO  ONE  ALPHA  proceeding  to  point  Whiskey,  ATC  issues 
clearance  to  intercept  and  fly  outbound  CHEUNG  CHAN  104°  radial. 
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DECISION  STRUCTURE 
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Tertiary  Choice:  1.  Maintain  configuration.  I  I- Connector 

2.  modify  configuration  to  optimize  performance.  _ _ 
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Captain  James  1.  Irving 


United  Air  Lines 


BOOTSTRAPPING  EXPERTISE: 

WHAT  MAKES  THE  EXPERT  PILOT  DECISION  MAKER 

EXPERT  AND  WHY  ? 


Conclusion: 

It  may  not  be  possible  or  prudent  to  forge  a  program  to  train 
decision  making  skills  if  we  have  not  derived  the  principles 
which  allow  us  to  answer  these  questions  — 

What  is  a  good  decision  ? 

How  is  if  arrived  at  ? 
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BOOTSTRAPPING 
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Jim  Irving,  Captain  737-30 
(My  views  are  my  own  and  are  not  necessarily— and 
probably  not— shared  by  my  employer,  United  Airlines.) 


(2)  Defining  the  expert  pilot  decision  maker  (3 )  Expert/Novice  Distinctions 

What  does  an  expert  have  beyond  average 

Traditional  definitions  of  “expert”  include  or  novice  ability?  When  faced  with  situations 
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(4)  Traditional  Efforts  to  Bootstrap  Expertise  (5)  Whats  wrong? 
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(6)  What’s  wrong?  What’s  wrong?  cont. 

2  -  SOPs  (7)  3 "  Training 
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(8)  4 -Practice  (10)  Pilots  Bootstrap  their 

•  on  single  problems,  rather  than  complex,  Decision  Making  Skills 
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(11)  Solutions?  (12)  Solutions?  cont. 

Suggestions  for  a  creating  a  program  for  4  -  Implementation  of  carefully  crafted 

training  expert  pilot  decision  makers  scenarios 
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(1 3)  The  Merits  of  ‘‘Hangar  Flying”  (1 4)  What  is  a  good  decision? 

How  is  it  arrived  at? 

•  shared  experiences 
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Dr.  Robert  A.  Alkov 


Nml  Safety  Center 


TRANS-COCKPIT  AUTHORITY  GRADIENT  IN 
NAVY/MARINE  AIRCRAFT  MISHAPS 


Conclusion: 

Wlhen  helicopter  pilots  of  unequal  rank  were  paired,  the 
potential  for  pilot  error  was  increased  over  the  potential  for 
those  crews  who  were  equal  in  rank. 
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Trans-Cockpit  Authority  Gradient  in  Navy/Marine  Aircraft  Mishaps 
Robert  A.  Alkov,  Michael  S.  Borowsky,  Dana  W.  Williamson,  and  David 

W.  Yacavone 
Naval  Safety  Center 
NAS  Norfolk,  VA  23511-5796 


Running  Head:  TRANS-COCKPIT  AUTHORITY  GRADIENT  IN  AIRCRAFT  MISHAPS 
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Abstract 

Navy  and  Marine  Corps  aircraft  mishaps  which  had  an  aircrew 
causal  factor  assigned  were  analyzed  to  determine  if  the  relative 
military  rank  of  the  pilot  and  copilot  or  Naval  Flight  Officer  was 
associated  with  the  rate  of  occurrence  per  100,000  flight  hours. 
All  class  A  and  B  helicopter  flight  mishaps  for  the  eleven  calendar 
year  period  1980-1990  were  examined.  Although  statistically 
significant  differences  were  not  found,  pairing  helicopter  pilots 
who  were  of  equal  rank  yielded  the  lowest  rate  (2.81),  seemingly 
refuting  Elwyn  Edward's  notion  that  a  flat  "trans-cockpit  authority 
gradient"  may  lead  to  greater  problems  in  the  cockpit  than  his 
hypothetical  "optimum  gradient".  When  there  was  one  rank 
difference  in  the  cockpit  the  rate  was  3.40.  When  copilots  flew 
with  pilots  who  differed  by  two  or  more  ranks,  the  largest  pilot 
error  rate  (3.45)  was  revealed.  These  findings  seem  to  support 
Edward's  hypothesis  that  a  steeper  "trans-cockpit  authority 
gradient"  may  be  detrimental  to  helicopter  flight  safety. 

Data  from  fighter  and  attack  jet  aircraft,  where  there  is  only 
one  pilot  flying  with  a  radar  intercept  officer  or  a  bombardier/ 
navigator,  were  also  studied  for  the  calendar  years  1986-91.  It 
was  discovered  that  the  lbwest  aircrew  error  rate  (1.80)  occurred 
when  the  pilot  and  the  other  aircrew  member  differed  by  more  than 
one  rank.  When  the  aircrew  were  of  the  same  rank  the  rate  was 
3.51.  For  those  crews  in  which  there  was  a  one  rank  difference  the 
rate  was  3.85.  Reasons  for  these  findings  are  discussed. 


NOTE: 

The  opinions  and  interpretations  expressed  are  those  of  the  authors 
and  should  not  be  construed  to  be  the  official  views,  policies, 
endorsements  or  decisions  of  the  Department  of  the  Navy,  the 
Department  of  Defense,  or  the  government  of  the  United  States  of 
America. 
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Trans-Cockpit  Authority  Gradient  in  Navy/Marine  Aircraft  Mishaps 

When  assessing  cockpit  performance  in  the  areas  of  aeronautical 
decision  making,  cockpit  resource  management,  and  maintenance  of 
situational  awareness,  interpersonal  dynamics  must  be  the  leading 
factor  studied.  The  dynamics  of  interaction  between  pilots  in  the 
cockpit  have  been  characterized  as  the  "trans-cockpit  authority 
gradient"  by  Elwyn  Edwards  (1988).  Edwards  believes  there  is  an 
optimum  gradient  to  allow  an  effective  interface  between  aviators. 
Although  he  does  not  provide  a  clear  definition  of  this  optimum,  he 
does  discuss  instances  where  it  is  not  achieved.  A  less  than 
optimum  gradient  occurs  when  the  pilot-in-command's  role  is  either 
over-emphasized  or  underplayed.  According  to  Hawkins  (1987), 
Edwards  feels  this  gradient  may  be  too  flat,  as  with  two  pilots  of 
equal  rank  and  qualifications,  or  too  steep,  where  the  pilot-in¬ 
command  is  a  domineering  senior  with  an  unassertive  copilot  (or 
vice-versa) .  The  steeper  gradient  can  occur  with  the  pairing  of 
pilots  with  copilots  of  either  higher  or  lower  rank,  experience, 
reputation,  qualifications  or  ability.  Edwards  also  reported  that 
the  chances  of  errors  going  undetected  and  uncorrected  increase 
when  this  gradient  is  not  optimized  (1988) .  Breakdowns  in  aircrew 
communications  leading  to  aircraft  mishaps  could  occur  as  a  result 
according  to  Foushee  (1982) .  Hawkins  (1987)  says  that  a  related 
problem  occurs  when  the  first  officer  advises  the  captain  of  an 
error  but  the  captain  fails  to  take  action.  According  to  Jensen 
and  Biegelski  (1989): 

...  a  less  technically  competent  crewmember  may  be  highly 
defensive  in  order  to  preserve  a  competent  self-image.  This 
may  result  in  the  crewmember  maintaining  unrealistic  and  self- 
deceptive  attitudes  of  personal  competence,  resistance  to 
stress,  and  a  lack  of  need  for  support  from  other  crewmembers. 
This  person  may  project  an  air  of  all-knowing  confidence  and 
independence  when,  in  fact,  the  opposite  is  true.  Such  behav¬ 
ior  may  have  a  highly  adverse  effect  on  CRM.  (page  177). 

A  study  of  249  airline  pilots  in  the  United  Kingdom  revealed 
that  40  percent  of  the  copilots  admitted  to  having  failed  to 
communicate  their  doubts  about  the  safety  of  the  flight  to  their 
captains  on  several  occasions.  Their  reasons  for  doing  so  ranged 
from  a  desire  to  avoid  conflict  to  deference  to  the  authority  of 
their  captains.  In  other  instances  the  first  officer  clearly 
advised  the  captain  who  failed  to  respond  or  take  action.  The 
first  officers  found  these  captains  to  possess  an  arrogant  and 
abrasive  manner,  with  strong,  intransigent  attitudes,  and  a 
domineering  style  of  work  interaction  (Wheale,  1983) . 

Data  from  the  Naval  Safety  Center's  files  reveal  that  during  the 
period  from  1980-90  aircrew  errors  accounted  for  58  percent  (175  of 
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304)  of  the  <^lass  A  and  B  helicopter  flight  mishaps  in  the  Navy  and 
Marine  Corps  .  Of  these  aircrew  error  mishaps,  57  percent  involved 
poor  aircrew  coordination  or  communication  (100  of  175).  Therefore 
33  percent  of  all  the  helicopter  mishaps  studied  involved  poor 
aircrew  coordination  or  communication.  In  order  to  expand  our 
earlier  study  of  helicopter  pilots  (Alkov,  et.  al.,  1992)  we  also 
included  fighter  and  attack  jet  aircraft  that  carry  only  one  pilot, 
but  with  another  non-pilot  aircrew  member  who  is  a  Naval  Flight 
Officer  (NFO)  .  We  focused  on  the  F-14  and  A-6  communities. 
Aircrew  causal  factors  were  present  in  32  percent  (103  of  323)  of 
the  mishaps  for  dual  seated  F-14  fighters  and  A-6  attack  bombers 
during  calendar  years  1980-90.  Poor  aircrew  coordination  was  a 
factor  in  35  percent  (36  of  103)  of  these  aircrew  caused  mishaps. 

The  total  of  all  Navy  and  Marine  Corps  helicopter  aircrew  error 
mishaps  accounted  for  218  fatalities  and  cost  414.7  million  dollars 
during  the  1980-90  period.  There  were  63  fatalities  and  costs  of 
1.19  billion  dollars  in  the  F-14  fighter  and  A-6  communities  during 
this  same  period. 

In  order  to  determine  if  there  is  an  optimum  gradient  of  cockpit 
authority  for  safety  in  Naval  aviation  operations,  we  undertook  an 
analysis  of  aircrew  error  mishaps.  To  ascertain  the  factors 
contributing  to  poor  aircrew  coordination  in  an  attempt  to  propose 
remedial  action  to  the  Navy,  we  examined  the  mishap  potential 
associated  with  the  relative  ranks  of  pilots  and  copilots  or  NFOs. 
We  wanted  to  test  Edwards  (1988)  assertion  that  the  ’’trans-cockpit 
authority  gradient "  may  be  too  flat  or  too  steep.  Therefore,  the 
null  hypothesis  of  this  study  was  that  pairings  of  pilots  with 
copilots  in  helicopters  and  pilots  with  NFOs  in  dual  seated  jet 
aircraft  would  have  the  same  aircrew  caused  mishap  rates  regardless 
of  rank  pairings. 


METHOD 

Navy  and  Marine  Corps  class  A  and  B  flight  mishaps  involving 
aircrew  factors  were  analyzed  in  an  attempt  to  refute  or  deny  the 
hypothesis.  The  flying  hours  of  Naval  aircrews  are  not  identified 


A  class  A  flight  mishap,  as  defined  by  the  Department  of 
Defense,  occurs  when  there  is  intent  for  flight  and  a  fatality  or 
permanent  total  disabling  injury  occurs;  or  the  total  cost  of 
property  damage  is  one  million  dollars  or  more;  or  an  aircraft  is 
destroyed.  A  class  B  flight  mishap  involves  property  damage  less 
than  a  million  dollars,  but  more  than  $200,000;  or  a  permanent 
partial  disabling  injury;  or  the  hospitalization  of  five  or  more 
persons.  A  class  C  flight  mishap  costs  more  than  $10,000  or  at 
least  one  lost  workday  due  to  injury. 
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as  to  the  rank  pairings  of  pilots  and  copilots  or  flight  officers. 
However,  aircraft  mishap  reports  contain  information  on  the  ranks 
and  flying  hours  of  the  aviators  involved.  In  order  to  estimate 
the  hours  flown  by  specific  pilot/copilot  or  pilot/flight  officer 
rank  combinations,  the  distribution  of  the  rank  combinations  found 
in  class  A,  B  and  C  mishaps  where  there  were  no  aircrew  causal 
factors  was  used.  This  method  enabled  the  flight  hour  estimates  to 
be  based  upon  the  largest  set  of  data  reported  to  the  Naval  Safety 
Center.  The  study  group  consisted  of  aviators  who  were  involved  in 
the  most  serious  aircrew-caused  mishaps  (class  A  and  B) .  The  data 
from  172  helicopter  mishaps  (3  of  the  original  175  lacked 
sufficient  information  for  analysis)  and  54  F-14  and  A-6  mishaps  in 
which  aircrew  error  was  assigned  were  analyzed  to  determine  if  the 
aircrew  members  were  of  equal  rank,  1  rank  removed  or  2  or  more 
ranks  removed  from  each  other.  The  number  of  serious  mishaps 
involving  each  rank  combination  was  then  divided  by  the  estimated 
flight  hours  per  100,000  for  each  combination  to  yield  an  aircrew 
error  mishap  rate  for  each  category  of  rank  pairing.  Helicopter 
pilot  pairings  were  placed  into  three  categories,  those  of:  (1)  the 
same  rank;  (2)  one  rank  different;  and  (3)  two  or  more  ranks 
different.  For  the  fighter/attack  crews  the  relative  rank  of  pilot 
and  NFO  was  also  examined.  Therefore,  five  combinations  were 
examined  for  these  aviators.  These  were  (1)  same  rank;  (2)  one 
rank  different,  pilot  senior;  (3)  one  rank  different,  pilot  junior; 
(4)  more  than  one  rank  different,  pilot  senior;  and  (5)  more  than 
one  rank  different,  pilot  junior. 

RESULTS 

The  aircrew  error  mishap  rate  for  serious  helicopter  mishaps 
(class  A  or  B)  was  2.81  for  the  group  of  pilots  and  copilots  who 
were  equal  in  rank.  When  pilots  and  copilots  differed  by  one  rank 
the  rate  was  higher  (3.40).  The  mishap  rate  increased  slightly 
(3.45)  when  pilots  were  flying  together  who  were  two  or  more  ranks 
apart.  The  Poisson  probability  distribution  did  not  reveal 
significant  statistical  differences,  (but  the  combined  rates  for 
pilots  who  differed  in  rank  was  higher  than  the  rate  for  the  equal- 
in-rank  group  (see  figure  1)).  This  finding  would  seem  to  be  of 
practical  importance. 


Insert  Figure  1  about  here 


As  for  the  combinations  of  pilots  with  NFOs  the  Poisson  Index  of 
Dispersion  revealed  no  statistically  significant  differences. 
However,  it  can  be  seen  in  figure  2  that  when  the  pilot  is  senior 
by  more  than  1  rank,  the  aircrew  error  mishap  rate  is  only  0.50. 
When  the  pilot  is  junior  by  more  than  1  rank,  this  rate  is  5.21. 
The  other  combinations  yielded  3.51  for  crewmen  of  the  same  rank, 
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3.75  when  the  pilot  is  one  rank  senior  and  3.97  when  the  pilot  is 
one  rank  junior  to  his  NFO. 


Insert  Figure  2  about  here 


DISCUSSION 

Is  mishap  potential  associated  with  the  ranks  of  helicopter 
pilots  and  copilots  flying  together?  Perhaps.  The  theory  that  a 
steep  trans-cockpit  authority  gradient  can  lead  to  poor  aircrew 
coordination  and  aircraft  mishaps  may  be  upheld.  However,  although 
too  flat  a  gradient  may  lead  to  problems,  these  problems  do  not 
occur  as  frequently  as  when  there  is  a  steeper  gradient.  Although 
there  are  documented  cases  of  low  ranking  pilots  flying  together 
having  mishaps  due  to  misunderstanding  as  to  who  was  in  charge, 
these  cases  are  rare.  Apparently  the  competition  factor  among 
peers  is  not  as  significant  to  flight  safety  as  is  a  steep  gradient 
of  authority.  This  is  somewhat  unexpected  due  to  the  aggressive 
and  competitive  nature  of  naval  aviators. 

The  mishap  rate  of  helicopter  pilots  who  are  paired  with 
copilots  whose  ranks  differ  is  greater  than  the  rate  for  pilots  who 
are  paired  with  copilots  of  the  same  rank.  According  to  Hawkins 
(1987),  "When  one  crew  member  monitors  or  has  reason  to  question 
the  data  used,  the  decision  or  performance  of  another,  the 
effectivity  of  the  monitoring  depends  on  the  response  which  it 
generates."  Obviously,  the  personality  of  the  senior  is  an 
important  factor.  If  the  response  is  to  ignore  the  input,  or  worse, 
to  react  negatively,  feedback  from  the  junior  will  decrease. 
Hawkins  goes  on  to  say,  "Leaders  in  all  walks  of  life  have  a 
tendency  to  reject  questioning  by  subordinates".  This  is 
especially  true  when  the  subordinates  are  low  in  the  aviation 
hierarchy.  Foushee  (1982)  found,  in  his  research  on  aircrew 
interaction,  that  among  airline  pilots,  subordinates  often  complain 
that  captains  can  be  so  insensitive  and  intimidating  that  their 
subordinates  are  hesitant  to  speak  up,  even  in  potentially 
dangerous  situations. 

The  most  junior  helicopter  pilots  are  typically  fresh  from  the 
Naval  Air  Training  Command,  where,  as  students  they  were  required 
to  defer  to  the  authority  of  flight  instructors  who,  for  the  most 
part  were  one  to  two  ranks  their  senior.  They  were  not  encouraged 
to  point  out  mistakes  to  these  instructors,  but  to  listen  to 
criticisms  of  their  own  flying.  Therefore,  they  are  not  very 
likely  to  be  assertive  in  the  cockpit  when  they  arrive  in  the 
fleet.  The  airline  pilot  survey  in  the  United  Kingdom,  referred  to 
earlier,  reported  that  captains  found  it  difficult  to  work  with 
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passive  copilots  (1983) . 

For  tb  i  fighter/attack  group  of  aviators,  the  low  aircrew  error 
mishap  rate  where  the  pilot  is  senior  to  the  NFO  by  more  than  one 
rank  is  interesting.  This  fact  and  the  higher  rates  that  were 
found  where  the  pilot  is  junior  to  the  NFO  by  more  than  one  rank 
may  be  reflecting  the  fact  that  pilots  who  are  less  experienced 
tend  to  have  higher  mishap  potential  than  experienced  pilots, 
regardless  of  whom  they  are  crewed  with. 

CONCLUSIONS 

When  helicopter  pilots  of  unequal  rank  were  paired,  the 
potential  for  pilot  error  is  increased  over  the  potential  for  those 
crews  who  were  equal  in  rank.  Senior  aviators  must  be  encouraged 
to  seek  feedback  on  flight  safety  information  from  their  junior 
aircrewmembers,  while  juniors  must  be  taught  to  be  more  assertive 
in  the  cockpit.  The  Naval  Safety  Center's  Aircrew  Coordination 
Training  Program  was  designed  to  address  these  issues.  A  study  we 
recently  completed  (Alkov  &  Gaynor,  1991)  documents  the  success  of 
this  program  in  reducing  aircrew  coordination  problems  in  naval 
aviation.  Thus,  although  a  less  than  optimum  trans-cockpit 
authority  gradient  will  continue  to  occur  on  the  flight  decks  of 
military  aircraft,  it  might  be  overcome  through  appropriate 
training. 

These  findings  raise  some  interesting  research  questions.  Is  the 
deferral  to  a  senior  pilot  caused  by  a  sense  of  inferiority  on  the 
part  of  junior  pilots  when  they  are  flying  with  seniors?  Is  there 
a  lack  of  self  confidence?  Does  the  difference  in  rank  obstruct 
their  working  as  a  team?  Is  this  an  inherent  characteristic  of 
military  life?  The  answer  to  these  questions  must  await  the 
results  of  ongoing  research. 

On  the  other  hand,  when  single-piloted  tactical  jet  aircrews  are 
examined,  the  relationships  between  the  pilots  and  their  non-pilot 
Naval  Flight  Officers  reveal  different  dynamics.  It  is  possible 
that  senior  pilots  flying  with  juniors  do  not  depend  as  heavily  on 
these  junior  NFO's  inputs  for  safety  of  flight  information.  They 
porbably  depend  on  their  NFOs  primarily  for  mission  performance. 
The  more  junior  pilots  are  less  experienced.  Their  higher  mishap 
potential  may  reflect  this  lower  level  of  experience.  As  the  only 
aviator  with  flight  controls  available  in  the  cockpit,  the  pilot  is 
solely  responsible  for  flying  the  aircraft.  Thus  the  mishap  rates 
of  inexperienced  pilots  may  be  higher,  regardless  of  the  rank  of 
their  NFOs,  because  of  their  lower  psychomotor  skill  levels. 
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Conclusion: 

Experts  build  a  mental  representation  of  meaningful 
relationships  in  a  situation.  These  relationships  are  more  than 
the  cognitive  knowledge  perceived  by  novices  in  the  same 
situation.  Long  familiarization  in  a  specific  field  of  knowledge 
transforms  the  experts  mental  representations  into  an  accessible 
form  of  "synthetic  global  knowledge"  which  when  applied  to 
working  situations  has  the  characteristics  of  instantaneous 
insight  or  intuition. 
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HOW  EXPERT  PILOTS  THINK 

Richard  J.  Adams  Vice  President 

Senior  Human  Factors  Engineer  Advanced  aviation  Concepts,  Inc. 


Research  on  expertise  and  the  differences  between  experts  and  novices  is  of  great  current  interest  and  rapidly 
expanding  into  many  areas  within  cognitive  psychology  and  cognitive  science  (Gordon,  1990).  Typically,  the 
research  approach  has  focused  on  expert  performance  in  academic  domains  such  as  geometry,  physics,  engineering 
mechanics,  etc.  and  employed  psychometric  testing  methods  to  explore  the  different  levels  of  cognitive  processing. 
There  is  a  real  scarcity  of  information  on  outstanding  individual  performance  or  expertise  in  applied,  real-world 
situations.  Yet,  this  type  of  performance  can  be  recognized  in  every  day  situations  as  diverse  as  financial  markets 
(Wall  Street  decisions),  emergency  response  medical  teams  (Chernobyl  physicians)  and  aviation  (the  United  Airlines 
flight  8 1 1  pilot/crew  performance). 

This  paper  provides  an  overview  of  the  conceptual  cognitive  psychology  research  that  defines  and  delineates  the 
important  characteristics  of  expertise.  It  is  hoped  that  the  understanding  of  the  cognitive  processes  associated  with 
"experts",  will  increase  the  awareness  of  these  processes  in  the  pilot  training  community.  The  importance  of  the 
attainment  of  a  higher  level  of  cognitive  skills  by  pilots  is  the  opportunity  to  enhance  performance  and  further  reduce 
human  error  accidents  through  improved  decision  making  training. 


Expertise 

In  virtually  all  arenas,  a  small  group  of  individuals  are  recognized  as  exceptional  performers.  The  abilities  of  these 
superior  performers  have  historically  been  assumed  to  be  the  results  of  natural  gifts  or  talent.  Most  of  the  research 
durinr  the  first  half  of  the  century  (Guilford,  1967;  Seashore,  1951  and  Tyler,  1965)  focused  on  the  identification  of 
these  individuals  with  specific  talents  prior  to  the  Stan  of  any  long-term  training.  For  most  domains,  this  type  of 
psych ometric  selection  had  very  limited  success  in  predicting  which  individuals  would  be  superior  or  outstanding 
after  training.  Research  in  the  last  twenty  years  (Chase  and  Simon,  1974;  Chi,  Glaser  and  Farr,  1988;  and  Lesgold, 
1984>  has  revealed  that  superior  performance  is  mostly  the  result  of  accumulated  skill  and  experience.  The 
primar /  differences  between  a  beginner  and  an  expert,  as  well  as  skilled  performance  at  different  intermediate 
levels,  can  be  attributed  to  acquired  knowledge  and  problem  solving  skills:  what  we  call  expertise. 

Expert  performance  can  be  generally  defined  as  the  selection  of  an  appropriate  response  to  situations  or  problems  in  a 
wide  variety  of  domains.  As  illustrated  in  Table  1 ,  these  might  include  selecting  the  best  move  in  a  chess  game, 
correc  ly  diagnosing  a  medical  problem,  or  using  the  proper  emergency  procedure  in  aviation.  The  relevant  research 
supporting  this  claim  has  focused  on  the  basic  understanding  of  knowing  how  to  respond  to  a  situation  rather  than 
knoH  ig  what  rule-guided  response  has  worked  in  the  past.  Intuition  or  know-how  refers  to  an  understanding  that 
effor'essly  occurs  due  to  discriminations  resulting  from  previous  experience.  Intuition  is  the  product  of  deep 
situational  awareness  and  involvement  quite  distinct  from  the  conscious  application  of  abstract  rules  (Dreyfus  and 
Dreyfus,  1986). 


During  the  last  decade,  expertise  has  been  studied  in  a  wide  range  of  domains  including:  medicine  (Patel  &  Groen, 
1991),  physics  (Anzai,  1991),  sports  (Allard  &  Starkes,  1991),  music  and  competitive  games  (Sloboda,  1991)  such 
as  chess  and  bridge.  From  this  extensive  research,  a  number  of  theories  of  expertise  have  evolved  (Adams  & 
Ericsson,  1992) 

The  current  theory  of  expertise  is  that  a  novice  first  solves  problems  by  weak,  domain  general,  heuristic  methods 
(often  working  backwards  from  the  goal);  successful  solutions  (when  repeated  frequently)  lead  to  the  development  of 
domain  specific  production  rules  and  the  beginnings  of  expertise;  as  these  rules  are  used  more  and  more  often,  and 
applied  to  many  situations  in  a  domain,  they  result  in  automatic  generation  of  specialized  productions  which  often 
use  forward  inferencing  to  progress  from  the  initial  problem  state  toward  a  solution  or  goal.  Relative  to  the  novice, 
the  expert  is  able  to  reach  the  correct  solution  more  quickly  and  efficiently. 
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Table  1  EXPERT  RESPONSES  IN  A  VARIETY  OF  DOMAINS 


DOMAIN 

SITUATION 

EXPERT  RESPONSE 

Chess 

A  specific  game  pattern 

Selection  of  the  best  move 

Physics 

A  difficult  problem 

Solution  generation 

Medicine 

Knowledge  of  a  patient’s  symptoms 
and  medical  history 

Correct  diagnosis  of  the  medical 
problem 

Machine 

Repair 

Description  of  equipment 
malfunction 

Correa  analysis  &  repair  of 
the  problem 

Aviation 

An  impending  emergency 

Application  of  trained 
procedures  or  generation  of 
appropriate  response 

Basic  Traits  of  Experts 

The  status  of  the  current  theories  of  expertise  are  thoroughly  summarized  in  "Thoughts  on  Expertise"  and  "On  the 
Nature  of  Expertise"  (Glaser  1986  &  1987).  The  latter  reference,  in  particular,  concludes  with  24  "Summary 
Propositions"  pertinent  to  this  analysis.  However,  rather  than  simply  restating  this  lengthy  list  or  reviewing  the 
entire  two  documents,  the  following  summary  of  findings  most  relevant  to  aviation  have  been  extracted. 

1.  Expert  performance  is  characterized  by  rapid  access  to  a  well  organized  body  of  conceptual  and 
procedural  knowledge.  Pilots  are  trained  and  tested  in  their  ability  to  perform  normal  and  “expected”  emergency 
procedures.  This  training  strengthens  and  expands  their  procedural  knowledge  base.  High  levels  of  competence 
result  from  the  interaction  between  knowledge  structure  and  processing  abilities. 

2.  The  organization  of  knowledge  used  by  experts  can  be  thought  of  as  schemata  or  a  modifiable 
information  structure  based  upon  knowledge  that  is  experienced.  This  includes  the  interrelationships 
among  objects,  situations  and  events  which  individuals  use  to  integrate  and  interpret  instances  of  related  knowledge. 
Schema  theory  assumes  there  are  schemata  for  recurrent  situations  that  expedite  decisions  in  certain  s  tuations  (e.g., 
the  experienced  pilot  anticipating  a  thunderstorm  by  recognizing  a  threatening  cloud  formation,  anticipating  wind 
shear  on  landing,  or  anticipating  in-flight  icing  conditions). 

3.  Expertise  is  domain  specific.  Within  a  domain,  experts  develop  the  ability  to  perceive  large 
meaningful  patterns.  Furthermore,  pattern  recognition  occurs  so  rapidly  that  it  appears  to  take  on  the  character 
of  insight  or  intuition.  This  ability  of  experts  appears  to  depend  on  the  nature  and  organization  of  knowledge  in 
existing  memory  which  is  directly  related  to  training  and  experience.  This  is  a  partial  explanation  of  how  expertise, 
while  domain  specific,  is  characterized  by  the  type  of  information  processing  of  data  within  a  domain. 

4.  Expert  knowledge  is  highly  procedural  and  goal  oriented.  Individuals  with  extensive  domain 
knowledge  are  much  better  at  relating  events  in  cause-and-e fleet  sequences  that  relate  to  the  goals  and  subgoals  of  a 
problem  solution. 

5.  The  capability  of  experts  to  fast-access  their  knowledge  facilitates  problem  perception  in  a  way  that  leads  to 
the  reduction  of  the  role  of  memory  search  and  general  processing.  Although  the  novice  and  expert 
have  equal  capability  for  cognitive  processing,  novices  typically  use  lots  of  search  and  processing  in  a  less  focused, 
more  general  manner.  The  outstanding  performance  of  experts  is  derived  from  how  their  knowledge  is  structured  for: 

•  Retrieval 

•  Pattern  Recognition 

•  Inference 
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This  expert  capability  is  also  referred  to  as  “holistic  discrimination  and  association”.  It  manifests  itself  in  the  ability 
to  intuitively  respond  to  patterns  without  decomposing  them  into  component  features.  This  understanding  occurs 
effortlessly  due  to  discriminations  resulting  from  prior,  concrete  experience. 

6.  Generalized  thinking  and  problem  solving  skills  may  develop  in  individuals  who  acquire  expertise  in  several 
domains  (e.  g„  aeronautics,  airplane  systems,  air  traffic  control  procedures,  emergency  procedures,  etc). 
Continuous  development  of  expertise  in  a  field  is  based  upon  novel  conditions  that  extend 
competence  to  novel  situations. 

7.  Experts  develop  specialized  schemata  that  match  goals  to  demands  of  the  problem.  Although  both 
novices  and  experts  can  display  good  use  of  general  problem  solving  process,  experts  use  them  primarily  in 
unfamiliar  situations. 

8.  The  development  of  expertise  is  influenced  by  task  demands  encountered  in  the  course  of  experience.  In 
some  domains,  experts  develop  the  capability  for  opportunistic  planning  which  enables  them  to  revise  problem 
representations  and  to  access  multiple  possible  interpretations  of  a  situation.  These  multiple  patterns  are  quickly 
assessed  and  used  to  develop  an  “internal”  visualization  and  then  create  a  goal  oriented  scenario  that  can  be  played  -- 
put  in  fine  detail  and  in  “slow-time”  —  to  a  successful  solution.  In  contrast,  novices  are  less  flexible  and  slower. 

Experts  build  a  mental  representation  of  meaningful  relationships  in  a  situation.  These  relationships  are  more  than 
the  cognitive  knowledge  perceived  by  novices  in  the  same  situation.  Long  familiarization  in  a  specific  field  of 
knowledge  transforms  the  experts  mental  representations  into  an  accessible  form  of  synthetic  global 
knowledge  (Bastic  1982)  which  when  applied  to  working  situations  has  the  characteristics  of  instantaneous  insight 
or  intuition. 

9.  Experts  also  possess  metacognition  abilities  that  are  not  present  in  less  experienced  decision  makers.  Experts 
develop  skilled  self-regulatory  processes  that  free  working  memory  for  higher  level  conscious  processing. 
These  capabilities  include:  planning  ahead,  efficiently  monitoring  one's  time  and  attentional  resources,  and 
monitoring  and  editing  one's  efforts  to  solve  a  problem.  Self-regulatory  activities  become  generalized  cognitive 
processes.  These  generalized  processes  become  important  when  an  individual  is  confronted  with  problems  in 
unfamiliar  areas. 

10.  An  important  point  of  distinction  is  that  there  are  both  routine  and  adaptive  experts.  Either  type  is  outstanding 
in  terms  of  speed,  accuracy,  and  automatic  cognitive  performance.  Either  type  can  construct  mental  models 
convenient  for  performing  their  tasks.  While  both  adaptive  and  routine  experts  are  very  confident  in  the  execution  of 
their  solutions,  routine  experts  have  somewhat  limited  capabilities  in  dealing  with  novel  or  new  problems.  Adaptive 
experts,  on  the  other  hand,  possess  the  ability  to  creatively  respond  to  novel  situations  and  develop  an 
appropriate  response  with  sane  reasonable  chance  for  a  successful  outcome. 

Routine  vs  Adaptive  Expertise 

The  distinction  between  routine  and  adaptive  experts,  points  to  an  avenue  for  the  next  generation  of  expertise 
research;  this  research  will  be  from  a  cognitive  psychology  perspective  and  will  stress  applications  to  real  world 
problems  like  how  aviators  respond  to  untrainable  emergencies.  A  broad  distinction  between  two  classes  of  expertise 
is  that  expert  performance  involves  "the  reliable  attainment  of  specific  goals  within  a  specific  domain"  (Sloboda, 
1991).  A  more  demanding  definition  is  that  "an  expert  is  someone  who  can  make  an  appropriate  response  to  a 
situation  which  contains  a  degree  of  unpredictability"  (Sloboda,  1991).  Perhaps  the  most  apt  general  characterization 
suggested  that  an  expert  is  someone  capable  of  doing  the  right  thing  at  the  right  time. 

In  general,  an  expert  will  have  succeeded  in  adapting  to  the  inherent  constraints  of  the  task.  If  the  task  can  be  done 
most  efficiently  by  forward  search,  the  expert  will  search  forward;  if  backward  search  is  better,  the  expert  searches 
backward.  If  certain  patterns  of  cues  are  crucial  to  performing  the  task  well,  the  expert  will  likely  perceive  and 
remember  them;  if  patterns  are  not  so  important,  the  expert  will  not  selectively  process  them.  The  tendency  of 
experts  to  adapt  to  task  constraints  would  account  for  the  fact  that  whereas  novices  differ  widely  in  the  way  they 
organize  domain  relevant  concepts,  experts  tend  to  resemble  each  other  (and  differ  from  novices)  in  their  conceptual 
organizations. 
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Time  Pressures 


The  adaptive  experts  ability  to  form  a  creative,  complete  and  integrated  representation  of 
complex  problems  is  critical  in  real-world  situations.  The  standard  experimental  paradigm  has  been  to 
present  a  situation,  e.  g.,  sheet  of  information  on  a  medical  patient,  a  chess  position,  a  scene  from  a  basketball 
game,  an  electronic  circuit  diagram,  etc.,  for  a  few  seconds  and  then  have  individuals  with  different  levels  of  expertise 
attempt  to  recall  as  much  as  possible.  Novices  can  recall  only  a  small  amount.  The  amount  of  recall  increases  with 
expertise  even  at  very  high  levels  of  expertise.  In  many  cases,  the  recall  of  the  super-expert  is  virtually  perfect. 
Expertise  in  sports  (basketball,  land  hockey,  etc.),  in  games  (chess  and  bridge)  and  in  computer  science,  electronics 
and  medicine  shows  that  the  validity  of  the  internal  representation  of  the  situation  increases  with  expertise. 

The  ability  to  internally  represent  external  situations  appears  to  be  related  to  the  skill  levels  that  experts  develop  to 
plan,  i.  e.,  mentally  explore  the  consequences  of  potential  sequences  of  actions  even  under  extreme  time  pressures. 
There  is  a  large  body  of  informal  evidence  suggesting  that  this  capability  to  plan  increases  with  the  acquisition  of 
expertise.  In  domains  where  there  is  a  pressure  to  respond  rapidly  often  in  response  to  dynamically  changing 
situations,  experts  develop  methods  of  coping  with  these  demands. 

The  shortest  possible  reaction  time  to  an  external  stimulus  even  after  extensive  training  is  about  200-250 
milliseconds,  and  more  complex  reactions  require  between  500-1000  milliseconds.  Research  on  elite  tennis  players, 
goal  keepers  in  hockey  and  baseball  players  has  shown  that  with  increasing  expertise,  individuals  dramatically 
improve  their  ability  to  correctly  predict  ball  or  puck  trajectories  relying  on  advanced  cues.  Similarly,  typists, 
pianists  playing  from  a  score,  and  individuals  reading  text  aloud  are  found  to  look  ahead  several  characters,  notes  and 
words  of  material.  This  is  referred  to  as  eye-hand  span.  The  size  of  the  eye-hand  span  is  closely  related  to  level  of 
expertise,  such  that  expert  performers  have  a  long  span  and  beginners  or  novices  have  essentially  no  span  at  all.  The 
relevance  of  the  eye-hand  span  in  aviation  is  that  it  allows  the  expert  pilot  to  scan  the  instruments  while  operating 
the  aircraft  controls,  tuning  radio  frequencies,  keying  the  microphone  to  talk  to  ATC,  reading  information  off  of 
charts,  etc.  In  short,  the  expert  pilot  operates  in  an  ultimate  "multi-tasking”  environment  and  needs  highly 
developed  eye-hand  span  capabilities  in  normal  workload  situations,  and  even  more  so  in  emergency  situations. 

These  expertise  domains  which  stress  speed  and  solutions  under  time  pressure  may  appear  to  always  be  driven  by 
automatic  responses  to  changes  in  the  current  situation.  However  the  ability  of  experts  to  anticipate  future 
conditions  (such  as  the  pilot's  need  to  change  aircraft  heading  or  call  ATC)  and  thereby  reduce  the  need  for  responding 
to  sudden  changes  in  the  situation  implies  an  important  role  of  an  internal  representation  of  the  situation  even  in 
these  types  of  expertise.  Furthermore,  the  portion  of  the  visual  field  that  can  be  clearly  seen  at  any  given  time  is 
only  a  fraction  (less  than  a  thousandth)  of  the  visual  field,  which  means  that  monitoring  the  visual  field  and  storing 
results  in  memory  for  later  use  and  updating  is  a  critical  part  of  having  an  accurate  description  of  the  situation. 

Expertise  and  Training  or  Practice 

At  the  most  general  level,  expert  performance  and  expertise  involves  the  acquisition  of  encoding  processes  allowing 
the  situation  (problem)  to  be  fully  represented  and  integrated  internally  in  such  a  way  that  relevant  actions  can  be 
retrieved  from  memory.  The  internal  representation  of  external  situations  is  also  critical  to  planning  and  evaluation 
of  possible  courses  of  action  as  well  as  a  means  to  represent  dynamically  changing  environment  for  the  purposes  of 
anticipation  and  prediction.  The  following  paragraphs  consider  how  this  form  of  expertise  can  be  attained  and 
promoted  by  training  and  instructional  activities. 

As  a  first  approximation,  acquisition  of  expertise  increases  linearly  in  all  relevant  aspects  of  performance  in  a 
specific  domain.  The  conventional  use  of  categories  to  describe  levels  of  expertise  or  phases  of  acquisition  of 
expertise  are  shown  in  Table  2.  Although  knowledge  about  how  experts  acquire  their  expert  performance  is 
relatively  limited,  generally,  the  novice  should  have  acquired  all  basic  knowledge  in  less  than  one  year.  In  parallel 
and  continuing  beyond  this  basic  knowledge  is  the  acquisition  of  problem  solving  skills  where  the  knowledge  is 
organized  to  effectively  produce  efficient  performance.  That  is,  there  is  an  acquisition  of  the  procedural 
knowledge  of  complex  patterns  occurring  in  specific  domains.  At  this  Intermediate  level,  differences  in  expertise 
appear  to  be  related  to  the  cued  recall  ability  and  the  number  and  complexity  of  those  patterns  available  for  use. 
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Table  2  PHASES  AND  CATEGORIES  OF  EXPERTISE 


PHASES  OF  EXPERTISE 


CATEGORY  OF  EXPERT 


Beginning  Phase  (Acquisition  of  declarative  knowledge 
and  domain  general  problem  solving  skills) 

About  1-2  years  of  active  experience  and  training 

Many  years  of  active  experience  and  training 
(Full  time  -  40-80  hours  per  week) 

More  than  10  years  of  full  time  experience 
and  training 


Beginner,  Student,  or 
Novice 

Intermediate 

Routine  Expert 
or  Journeyman 

Master  or  Adaptive 
Expert 


Finally,  in  both  the  routine  expert  and  adaptive  expert  categories,  an  accepted,  domain  specific  vocabulary  (or  jargon) 
is  developed  to  allow  efficient  communication  among  experts  and  masters  in  a  given  domain.  This  is  obvious  in 
medicine  or  law  which  also  involve  the  use  of  Latin,  French  and  to  a  lesser  degree  German.  Similarly,  in  aircraft 
operations  (from  flight  planning  to  air  traffic  control)  experts  have  developed  an  extensive  jargon  which  is  formalized 
in  the  “Pilot-Controller  Glossary”  of  the  Airman's  Information  Manual.  This  manual  is  designed  to  promote 
efficient  communications  and  a  common  understanding. 

Most  of  our  knowledge  about  how  expert  performance  is  attained  comes  from  highly  competitive  domains  with 
relatively  vigorous  evaluation  procedures  such  as  sports,  games  and  music.  Biographical  analyses  of  the  international 
level  performers  in  these  domains  show  that  they  start  early.  By  the  age  of  seven  most  of  them  are  engaged  in 
instruction  and  daily  practice.  The  amount  of  practice  is  gradually  increased  to  about  two  to  four  hours  daily  during 
the  early  to  late  teens  respectively.  During  early  adulthood,  these  individuals  spend  virtually  all  their  time  on 
activities  related  to  their  domain  of  expertise.  This  amount  of  practice  appears  to  be  the  single  most  important 
variable  in  determining  the  attained  level  of  performance  in  a  given  domain.  Athletes  and  expert  musicians  clearly 
distinguish  practice  as  the  most  important  activity  for  further  improving  performance. 

Pilots,  on  the  other  hand,  must  practice  for  events  that  most  likely  will  never  occur.  This  creates  a  different  expert 
training  or  development  scenario.  For  example,  attainment  of  expert  status  in  aviation  and  other  domains  such  as 
architecture,  engineering  and  medicine  appears  to  be  the  result  of  a  slow  accumulation  of  experience  in  “on-the-job” 
environments.  Given  the  relatively  unstructured  nature  of  this  “practice”  and  the  relative  infrequency  of  objective 
evaluation  of  performance  or  guidance  by  a  master  instructor,  improved  performance  and  expertise  in  these 
fields  relies  much  more  strongly  on  the  motivations  of  the  individual. 
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Can  we  develop  an  expert  level  decision  making 
training  syllabus? 
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But,  What  is  going  on  at  the  cognitive  level??? 


81 


AAC 


82 


o 

< 

< 


CO 

■  MB 

o 

<D 

CL 

CO 

CO 


C  “D 

—  <D 

C  CO 
O  3 

H 

E  o 

■£S 

•RiS 

O  m 

+*  co 

c  £ 
3  CO 

O  d> 


CO 

■u 

c 

o 

o 

CD 

CO 

O 

CM 

i 

00 


E 

o 

t: 

o 

CO 

o 


CD 

'55 

c 

CO 


m 


O 

sa 

O 

CO 

E 

CD 


CM 

+1 


E  co  L 

EuT§ 

®§0 

+■*  U  nr 

O)  -* 

C  DC 


O 

c 


8JJ 

K 

< 


CO 


o  CD 

y.  o 


CO 


—  r'  ^ 

75  <J  n 

CD  O^Uj 

2  wp 

ts  oC. 

w-Q  « 
.IS  fl»  P 

=  :5,o 


CD 


■u  X 

g  § 

3  co 

_:  T3 


co 


w  35 


o 
3 

i? 


0  ®  11  ^  CO  CO  < 

“  —  co  - 


c  ** 

ZZ  m 


o 


83 


84 


CONCEPTS 


o 

< 

< 


85 


87 


AAC 


Dr.  K.  Anders  Ericsson 
Associate  Professor 


Dept,  of  Psychology 
U.  of  Colorado,  Boulder 


methodology  for  studying 

AND  TRAINING  EXPERTISE 


Conclusion: 

Expertise  in  aviation  provides  the  ability  to  avoid  emergency 
situations  as  well  as  the  readiness  to  respond  to  both  normal 
and  abnormal  situations  rapidly.  Realistic  training  scenario 
development  to  achieve  these  goals  will  require  substantial 
adaptation  and  testing  to  yield  effective  results. 
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METHODOLOGY  FOR  STUDYING  AND 
TRAINING  EXPERTISE 

by 

Anders  Ericsson,  Institute  of  Cognitive  Science 
and  Department  of  Psychology 

University  of  Colorado  at  Boulder 


I  Brief  Historical  Background  for  the  Study  of 
Eminent  People  and  Experts: 

Some  Approaches  and  Issues 

II  The  Study  of  Superior  Expert  Performance 

III  Sketch  of  a  Proposal  for  Diagnosis  and 
Training  of  Expert  Decision-Making  in  Pilots 
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I  Brief  Historical  Background  for  the 
Studvof  Eminent  People 
and  Experts: 

Some  Approaches  and  Issues 


Eminent  individuals  and  experts  differ  from  average 
people  in  their  abilities  and  achievements 


-  Identification  of  exceptional  individuals  using 
social  criteria  of  success 


Awards,  Occupational  status,  salaries 

Individual  differences  in  basic  capacities  and 

characteristics 

vs 

Differences  in  acquired  level  of  skill 
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T  alent 
view 

Avoid 

influence  of 
experience 


Skill 

acquisition 

view 

Maximum 
influence  of 
experience 


Basic 

processes 

Speed  of  neural  process 
(simple  reaction  time) 


Acquired 

characteristics 

Speed  of  reacting  in 
typical  situation 


Characteristics  of  basic  Memory  for  repre¬ 
memory  processes  sentative  stimuli 

Physical  and  anatomical 
characteristics 

•  compositions  of  muscles 

•  capillaries  and  hearts 


Individual  differences  in  basic  capacities 


Conclusions: 

No  reliable  differences  between  Experts  and  Novices 
(Exception:  height  differences  in  sports) 

Attempts  to  predict  ultimate  performance 

after  training  have  failed 
(less  than  4%  of  variance  accounted  for) 

Large  effects  of  training  for  performance 

on  tasks  measuring  basic  abilities. 


Superior  reaction  times  and  memory  performance 
for  tasks  in  the  domain  of  their  expertise. 
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Studying  instances  of  superior 
achievements  and  performance 


Situation  &:  Individual  — >  Outstanding  Achievement 

Recency 

Regularity 

Reproducibility 


Standardized  situations 

Sports  and  other  competitive  events  (chess,  bridge) 
Natural  competition  (stock  market) 

Conclusions 

Some  forms  of  expertise  are  difficult  to  measure  and 
study 

Expert  status  not  always  associated 
with  superior  performance 
Computer  programmers 
Expert  decision-making 
Applied  mathematicians 
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Outstanding  Performance 


individual 

interaction 


discoveries 
military  and 
political 
decisions 


discoveries 
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unique 

individual: 


winners  of  fair 
competitions 


II  The  Study  of  Superior  Expert 
Performance 


Capturing  superior  expert  performance 

Essence  of  superior  performance  in  a  domain  of 
critical  taste  -  Task  analysis 
Laboratory  analogues 
Analysis  of  superior  expert  performance 

Process-tracing  (verbal  reports,  reaction  time) 
Experimental  variation  and  analysis 
Findings:  Rapid  understanding  &  intuitive  reactions 

Slower  systematic  evaluation 
Superior  memory  for  relevant  information 
Analysis  of  acquisition  and  maintenance  of  expert  performance 
Deliberate  practice  rather  than  experience 

Necessary  amounts  of  deliberate  practice  for 
attaining  expert  performance. 

Constraints  on  daily  and  weekly  amounts  of 
deliberate  practice 
Summary 
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tennis  player  serving  where  the  ball  will 
a  ball  towards  the  land  within  the 
camera>  service  area 


De  Groot's  Task: 


Select  the  Best  Move  for  a  Given  Chess  Position 
while  Thinking  Aloud. 


POSITION  A 


White  on  move 
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Organization  of  Expert  Performance 


I.  Traditional  View  {Pattern— ^Action} 


Situation 


>  Patterns - >  Action 


II.  Revised  View 

•  Memory  for  briefly 

presented  information 

•  Mental  exploration  of 

possibilities  (Planning) 

•  Evaluation  and  reasoning 


•Action 
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ennis  Repetitive  practice  Competition  Recreate 

with  a  coach  on  play 

perfecting  a  specific 

shot 


Year  at  which  Transition  to 

Practice  was  a  Full-Time 

Initiated  Involvement 
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Frequency 
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Sports:  Differences  between  athletes 
at  different  levels  of  performance 

•  Earlier  ages  for  start  of  practice 


•  More  weekly  practice 


Note:  Similar  amounts  of  daily  practice 
for  elite  athletes  and  elite  musicians 
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Ill  Sketch  of  a  Proposal  for  Diagnosis  and 
Training  of  Expert  Decision-Making  in  Pilots 

Commercial  flying  vs.  military  combat 

Expertise  in  commercial  flying 

Readyness  to  respond  to  emergency  situations 
Ability  to  avoid  emergency  situations 

General  design  of  tasks 

Relevant  information  about  situation 
Perceptually  available  information 

Presentation  of  information  about  emergency  conditions 

Timed  generation  of  actions  or  plan  for  action 
Retrospective  report 
Recall  of  all  information  about  situation 

Training  of  experts  in  other  domains 

Chess  &  Bridge  -  Study  of  master  games 
Medicine  -  Rounds,  case  conferences 
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Richard  J.  Adams 


Sr.  Human  Factors  Engineer 
Advanced  Aviation  Concepts 


ENHANCED  ADM 
TRAINING  ALTERNATIVES 


Conclusion: 

Expert  cognitive  performance  is  characterized  by  rapid  access  to 
a  well  organized  body  of  conceptual  and  procedural  knowledge. 
This  is  a  modifiable  information  structure  based  upon 
knowledge  that  is  experienced.  This  experience  allows  the 
perception  of  large  meaningful  patterns  in  familiar  and  new 
situations  which  help  the  expert  match  goals  to  task  demands. 
This  means  they  can  respond  creatively  or  with  opportunistic 
solutions  when  the  situation  requires  such  a  response. 
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ENHANCED  ADM  TRAINING  ALTERNATIVES 


Richard  J.  Adams  Vice  President 

Senior  Human  Factors  Engineer  Advanced  aviation  Concepts,  Inc. 


The  overview  of  this  field  of  research  has  shown  that  the  development  of  expertise  relies  heavily  on  training  and 
requires  considerable  amounts  of  experience  in  a  specific  field.  Furthermore,  experts  rely  on  a  wide  variety  of 
different  processing  skills  and  unique  problem  solving  capabilities.  As  summarized  in  Gordon  (1990): 

1 .  Experts  have  more  detailed,  better  organized  knowledge  structures. 

2.  Experts  perceive  and  organize  problems  on  a  more  abstract  level  than  novices. 

3.  Experts  perceive  problems  in  large  meaningful  patterns  related  to  the  context. 

4.  Experts  are  much  faster  than  novices  because  of  their  use  of  procedural  knowledge  and  forward  inferencing 
techniques. 

All  of  these  characteristics  are  equally  applicable  in  the  expert  pilot  domain.  At  the  same  time,  the  past  IS  years  of 
aviation  research  in  Aeronautical  Decision  Making  (ADM)  has  developed  training  manuals  which  teach  pilot 
judgment  as  a  two  step,  linear  process  (Jensen  and  Benel,  1977): 

•  The  ability  to  search  for  and  establish  the  relevance  of  all  available  information  about  oneself,  the  aircraft,  the 
environment,  the  flight  situation;  to  specify  alternative  courses  of  action;  and,  to  determine  expected  outcomes  from 
each  alternative. 

•  The  motivation  to  choose  and  authoritatively  execute  a  course  of  action  which  assures  safety  within  the  time  frame 
permitted  by  the  situation. 

Although  this  ADM  training  program  has  been  successful  in  decreasing  the  number  of  accidents  and  incidents  in  the 
inexperienced  pilot  group  (less  than  five  years),  it  has  not  been  as  successful  with  the  more  experienced,  high  time, 
expert  pilot  group  (Albert  1989,  Alkov  1991).  That  is,  their  accident  rate  (accidents  per  100,000  flight  hours)  has 
not  been  affected.  At  least  one  reason  for  the  apparent  shortcomings  of  ADM  training  is  that  it  teaches  a  linear, 
algorithmic  process  of  controlled  decision  making  that  is  not  generally  the  way  people  make  decisions,  especially 
experts,  and  especially  not  in  emergency  or  stressful  situations. 

In  fact,  a  review  of  aviation  examples  (Adams  &  Ericsson,  1992)  where  expert  pilots  “saved  the  day”  either  in  whole 
or  in  part,  documented  that  pilot's  making  decisions  under  stress  exhibit  five  basic  characteristics  shown  in  Table  1. 


Table  1  CHARACTERISTICS  OF  EXPERT  PILOTS 


[•  RE' 
•  INS 
•  RE/ 
•  PO< 


REVERSION  TO  BASIC  AIRMANSHIP  SKILLS 
INSTANTANEOUS  RECALL  OF  TRAINING 
REASONED  APPROACH  IN  EMERGENCIES 
POSITIVE  IN  APPROACH  &  EXPECTATIONS 
SELF-ASSURED  AND  OPTIMISTIC 


The  following  discussion  explores  the  development  of  these  expert  pilot  characteristics  and  attempts  to  relate  that 
development  to  conventional  training,  experience,  cognitive  processing  development  and  the  new  directions  or 
innovations  required  for  further  improvements  in  decision  making  training. 
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Stages  in  Development  of  Pilot  Cognitive  Processes 

For  the  purposes  of  the  following  discussion,  it  is  important  to  note  that  human  cognition  is  task  dependent  and 
purposeful  (goal  oriented).  That  is,  humans  use  their  knowledge,  cognitive  processing  skills  and  the  cues  or  stimuli 
of  a  situation  or  task  to  develop  problem  solving  approaches.  To  accomplish  this,  two  types  of  knowledge  are  used. 
These  arc  declarative  knowledge  and  procedural  knowledge.  Declarative  knowledge  consists  of  knowledge  that  can  be 
verbalized,  some  call  this  knowledge  about  “facts  and  things".  Procedural  knowledge  is  knowledge  about  actions  or 
how  to  perform  various  cognitive  activities.  These  very  often  cannot  be  completely  or  adequately  verbalized;  for 
example,  how  to  ride  a  bike  could  be  broken  down  and  described  but  the  result  would  lose  the  essentials  of  the  “how¬ 
to”.  However,  procedural  knowledge  is  the  basis  for  development  of  specific  steps  (also  called  production  rules)  to 
be  used  in  problem  solving  situations.  As  experience  is  gained,  pilot’s  rely  more  and  more  on  the  use  of  procedural 
knowledge  to  solve  problems.  Furthermore  they  solve  problems  with  increasing  speed  and  accuracy  using  this  type 
of  knowledge  as  shown  in  Figure  1 . 

INCREASING  DECISIONAL  SPEED  AND  ACCURACY 


1.  COGNITIVE 

Figure  1  EVOLUTION  OF  EXPERT  KNOWLEDGE  TYPES  AND 
PROCESSING  TYPES 

Cognitive  psychology  recognizes  three  stages  in  the  development  of  expert  problem  solving  skills  (Anderson  1985) 
These  arc  cognitive,  associative  and  autonomous  During  the  first,  cognitive  stage,  pilots  commit  to  memory  a  set 
of  facts  relevant  to  a  desired  skill.  They  typically  rehearse  these  facts  as  they  first  perform  the  skill.  For  example, 
novice  pilots  learning  stall  recovery  will  memorize:  recognize  the  stall,  lower  the  nose,  apply  full  power,  level  the 
wings  and  minimize  altitude  loss.  In  this  stage,  they  arc  using  their  general  aeronautics  knowledge  (domain-general) 
to  guide  their  solution  to  loss  of  lift  over  one  wing,  and  solve  a  domain  specific  problem,  how  to  keep  the  aircraft 
flying.  The  problem  solving  capabilities  and  level  of  expertise  in  this  stage  are  very  basic.  Novices  spend  a  lot  of 
time  searching  and  moving  around  declarative  knowledge. 

The  second,  or  associative  stage,  has  two  important  characteristics.  First,  errors  in  the  initial  understanding  and 
performance  arc  detected  and  gradually  eliminated.  The  novice  pilot  learns  to  coordinate  the  nose  drop,  power 
application  and  rudder  application  for  a  smooth  stall  recovery.  Second,  the  connections  between  the  various  elements 
required  for  successful  performance  are  strengthened.  The  pilot  does  not  sit  for  a  few  seconds  trying  to  decide  which 
action  to  perform  first  after  lowering  the  nose.  Basically,  the  outcome  of  the  associative  stage  is  a  learned 
procedure  or  production  rule  for  performing  a  desired  response  to  a  known  situation.  In  this  stage,  the  declarative 
information  is  transformed  and  integrated  into  a  procedural  form.  However,  the  procedural  form  does  not  necessarily 
replace  the  declarative  knowledge.  Rather  the  two  forms  coexist  and  are  available  when  needed  for  the  task.  For 
example,  the  low  time  pilot  can  fly  the  airplane  while  simultaneously  talking  to  ATC  and  navigating.  AH  the 
while,  he  still  remembers  the  rules  of  aerodynamics,  the  characteristics  of  a  stall  and  the  recovery  process. 


Ill 


The  third  cognitive  suigc  occurs  when  die  problem  solving  procedures  become  faster  and  more  automated.  There  is 
not  necessarily  any  sharp  distinction  between  the  associative  and  autonomous  stages  of  expertise.  Rather,  the 
autonomous  stage  evolves  from  the  repeated  application  of  known  patterns  and  their  associative  use  to  achieve 
solutions.  The  use  of  declarative  knowledge  or  “verbal  mediation”  often  disappears  during  this  stage  of  cognitive 
processing,  at  least  for  some  tasks.  In  fact,  the  ability  to  verbalize  knowledge  of  the  procedure  can  be  lost. 
Furthermore,  expert  cognitive  process  development  appears  to  develop  continually  in  a  specific  area  or  domain. 
Throughout  the  development,  the  skill  gradually  improves.  Ultimately,  the  skill  can  be  extended  to  the 
ability  to  respond  to  cues  not  previously  encountered  and  to  develop  new  solutions  or 
production  rules  applicable  to  novel  situations.  The  refinement  of  the  expert  pilot's  cognitive  processing 
and  the  characteristics  associated  with  the  three  stages  are  illustrated  in  Figure  2. 


1.  COGNITIVE 


A.  Linear  processing 
(checklist) 

B.  Controlled  attention 

C.  Learned  problems 
&  Hueristic  decisions 


2.  ASSOCIATIVE 

A.  Parallel  processing 
&  pattern  recognition 

B.  Dynamic  thinking 

C.  Production  rules 

&  procedural  decisions 


3.  AUTONOMOUS 

A.  Creative  problem  solving 
(opportunistic  planning) 

B.  Forward  Inferencing 

C.  Judgment  and  Intuition 
or  Insight 


Figure  2  CHARACTERISTICS  OF  EXPERT  COGNITIVE  PROCESSES 


Pilot  Training  and  Information  Processing 

Conventional  pilot  training  has  been  based  upon  this  foundation:  factual  or  declarative  knowledge;  flying 
procedures  development;  and,  basic  pilot  skills  or  abilities  as  shown  in  Figure  3.  The  novice  pilot,  therefore,  is 
expected  to  leant:  aerodynamics,  airplane  performance  capabilities  and  limitations,  electrical  and  hydraulic  systems. 
Federal  Aviation  Regulations,  etc.  He  is  then  trained  in  aircraft  control  and  operation  for  both  normal  and  emergency 
situations.  This  training  includes  a  procedures  development  for  prefiight,  takeoff,  cruise,  approach  and  landing 
phases  of  flight.  Through  this  training,  the  novice  develops  and  improves  his  basic  psychomotor  abilities  and  hones 
his  flying  skills.  At  this  stage,  judgment  or  decision  making  is  only  taught  informally  through  training  session 
debriefs,  hangar  flying,  analyses  of  other  pilots  experiences  and  the  limited  flight  experience  gained  in  preparation  for 
an  airman  certification  test.  After  successfully  passing  the  test,  the  novice  pilot  is  expected  to  cautiously  begin 
developing  good  decision  making  and  judgment  skills  as  he  gains  experience.  This  provides  the  basis  for  the 
development  of  more  sophisticated  judgment  as  experience  is  gained. 

In  aviation,  training  is  highly  procedure  oriented  both  in  developing  flying  skills  (psychomotor)  and  in  decision 
making  skills  (cognitive  and  informational)  for  normal  and  emergency  operation  of  the  aircraft.  These  procedures  and 
skills  provide  the  foundation  for  the  development  of  more  sophisticated  production  rules  (procedural  knowledge)  as 
experience  is  gained. 

The  newly  qualified  or  low  time  aviator  (100-1000  hours)  develops  his  flying  and  decision  making  skills  through  1 
to  5  years  of  experience.  This  experience  allows  him  to  expand  his  procedural  knowledge  base  using  encounters  with 
real-world  problems  and  operational  constraints.  At  the  same  time,  his  decision  making  ability  and  cognitive 
processing  is  strengthened  by  repeated  use  of  trained  patterns  and  expanded  associative  networks  of  experience 
patterns.  The  low  time  pilot  is  at  the  second  stage  of  cognitive  process  development;  he  has  begun  to  develop  the 
speed  and  quality  of  processing  of  the  Routine  Expert. 
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MORE  EXPERIENCE 
AND  TRAINING 


NOVICE  LOW  TIME  EXPERT 

PILOT  PILOT  PILOT 

Figure  3  CONVENTIONAL  PILOT  JUDGMENT  DEVELOPMENT 

Finally,  the  Expert  Pilot  (1000-10,000+  hours)  mainly  relies  on  automatic  cognitive  processing  abilities.  Just  as  in 
the  other  domains  of  sports,  games,  music,  and  medicine,  the  Expert  Pilot  has  achieved  a  tremendous  base  of 
procedural  knowledge  and  skills  applicable  to  normal  day-to-day  flying  problems,  trained  emergencies  (such  as  an 
engine  failure)  and  novel  or  untrainable  emergencies.  He  uses  this  procedural  knowledge  base  for  a  very  high 
percentage  of  his  problem  solving  and  decision  making  just  as  the  Routine  Expen  in  other  domains.  In  addition,  he 
has  the  similar  routine  ability  to  retrieve  and  integrate  information  from  his  declarative  knowledge  base,  if  the 
situation  requires  that  action. 

The  review  of  the  case  studies  of  expen  pilot  performance  presented  in  Adams  &  Ericsson,  1992  as  well  as  NASA 
aviation  research  (Chappell  1991,  Degani  1991)  indicate  that  experienced  pilots  exhibit  the  “typical”  characteristics 
of  expen  cognitive  processes.  Figure  4  illustrates  the  relationships  between  levels  of  pilot  experience,  types  of 
knowledge  used  for  problem  solving  and  the  three  stages  of  development  of  cognitive  processing  ability. 


TYPE  OF  PROCESSING 


Figure  4  EXPERT  PILOT  JUDGMENT  DEVELOPMENT 
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As  shown  in  the  figure,  one  main  characteristic  of  the  development  of  expert  cognitive  processes  is  the  continual 
increase  in  decisional  speed  and  accuracy  as  experience  is  gained  in  a  specific  area,  e.g.,  aviation.  In  fact, 
these  two  characteristics  are  precisely  the  areas  of  decision  making  and  problem  solving  most  affected  by  experience 
and  training  or  “practice”. 

The  three  types  of  cognitive  processing  in  the  development  of  the  expert  pilot  correspond  to  an  increased  use  of 
procedural  knowledge  gained  from  both  experience  and  training.  The  expert’s  cognitive  processing  which  relies  on 
recognizing  known  patterns  and  solving  problems  with  automatic  use  of  production  rules  or  procedural  knowledge 
corresponds  to  the  conventional  development  from  novice,  to  low-time,  to  expert  pilot  from  a  flying  skills  and 
aviation  procedures  perspective.  This  is  an  extremely  valuable  finding  since  it  facilitates  the  understanding  of  the 
developmental  relationship  of  cognitive  skills  with  the  development  of  aeronautical  or  aviation  skills.  In  fact,  it 
could  be  argued  that  pilot  training  has  included  “expert”  cognitive  process  training  all  along 
simply  as  a  result  of  the  strong  emphasis  on  aviator  procedures. 

To  summarize:  the  novice  or  ab  initio  pilot  responds  (cognitively)  to  stimuli  or  external  cues  based  upon  a 
thorough  understanding  of  a  complex,  declarative  knowledge  base.  His  decisions,  whether  normal  or  critical,  are 
typically  based  on  a  linear  problem  solving  approach  (some  type  of  checklist  or  “DECIDE”  type  of  model).  His 
capabilities  are  generally  limited  to  the  procedures  he  has  learned  and  expedited  by  the  use  of  rules-of-thumb  (or 
heuristics).  This  type  of  cognitive  judgment  is  somewhat  restrictive,  but  usually  successful,  in  its  application  to 
familiar  tasks  or  problems.  The  novice  is  aware  of  the  situational  demands  and  reacts  or  responds  to  ti.e>-.  but  with 
limited  cognitive  and  analytical  resources. 

The  low  time  pilot  or  (associative  problem  solver)  has  the  capability  for  an  enhanced  decision  making.  As  a  result 
of  his  experience,  additional  flight  training  and  possibly  a  knowledge  of  ADM  principles,  the  pilot  develops  a 
capacity  for  more  dynamic  cognitive  processing.  At  the  associative  level,  he  stores  information  in  terms  of 
schemata  which  are  modifiable  information  structures  based  upon  experience.  The  associative  pilot  uses  pattern 
recognition  and  dynamic  interrelationships  among  objects,  situations  and  events  to  integrate  and  interpret  related 
knowledge  instead  of  the  static,  linear  thinking  of  the  novice.  This  level  of  cognitive  processing  is  characterized  by 
the  early  development  of  the  capabilities  of  a  routine  expert  in  that  certain  large  patterns  are  spontaneously 
recognized  rather  than  requiring  a  conscious  search  of  declarative  knowledge  and  a  checklist  review  or  alternative 
solutions.  The  associative  thinker  is  in  the  process  of  evolving  into  an  expert  in  the  general  sense  of  his  procedural 
knowledge  and  use  of  production  rules,  and,  as  stated  earlier,  it  is  difficult  to  draw  a  specific  line  of  demarcation 
between  associative  and  automatic  problem  solving. 

The  Expert  Pilot 

The  Expert  Pilot  is  “adaptive”.  In  addition  to  having  all  the  traits  gained  through  experience  and  training,  he  can 
alter  his  procedures  in  real  time  (modify,  delete  or  expand).  He  can  create  new  rules  and  patterns  based  upon  unique, 
previously  unencountered  problem  characteristics.  This  capability  to  creatively  respond  u  unique  problems  or  novel 
task  demands  identifies  the  highest  level  of  expert  pilot  cognitive  processes.  In  fact,  the  expert  pilot  s  ability  to 
adapt  to  task  demands,  set  goals  and  retrieve  solutions  from  memory  occurs  so  rapidly  it  appears  to  be  intuitive 
problem  solving  in  many  instances. 

This  “adaptive”  capability  is  referred  to  as  “KNOWING  WHEN”  (Dreyfus  and  Dreyfus  1986).  That  is,  the 
Adaptive  Expert  Pilot  perceives  the  necessity  to  alter  ingrained  procedures  based  upon  the 
parameters  and  dynamics  (or  cues  and  context)  of  the  problem  or  situation  encountered.  It  is 
believed  that  this  “KNOWING  WHEN”  (an  almost  direct  perception  of  the  proper  course  of  action)  may  provide  the 
key  to  the  next  generation  of  ADM  training.  This  would  require  a  training  environment  or  “situation”  that  provided 
the  necessary  cues  and  context  to  trigger  the  expert's  adaptive  processing  mechanism.  Replicating  the  inflight 
kinesthetic  cues  using  a  simulator  and  the  pilot  workload  cues  using  typical  emergencies  or  Line  Oriented  Flight 
Training  (LOFT)  scenarios  may  not  be  sufficient  to  trigger  the  associative  mental  “hooks”  or  adaptive  cognitive 
process. 

Since  experts  store  information  as  schemas  or  organized  sets  of  facts,  relationships  and  perceptions,  these  same 
schemas  are  a  “major  mechanism”  (Anderson  1985)  for  problem  solving  either  in  the  simulator  or  inflight. 
Therefore,  the  retrieval  of  information  and  problem  solving  procedures  will  improve  the  more  closely  the  cues  and 
context  during  training  match  the  real  “experience".  The  expert  pilot’s  perception  of  the  whole  situation  involves  a 
sense  of  relations  that  include  physical,  cognitive  and  internal  effects  which  are  used  to  both  store  and  retrieve 
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knowledge.  This  is  what  is  meant  by  the  term  “synthetic  global  knowledge”  (Bastic  1982)  used  by  experts  for 
“opportunistic  planning”. 

The  coordinated  use  of  cues  and  context  with  stored  schema  is  believed  to  be  a  “major  mechanism”  used  by  experts 
to  infer  unobserved  or  unknown  elements  of  a  problem  in  “knowledge-lean”  or  “untrainable”  novel  situations. 
Delving  into  how  experts  develop  insight  into  causal  relationships  in  current  situations  by  applying  their  global 
knowledge  requires  further  understanding  of  the  role  of  experience  in  cognitive  processing  skills  development. 

The  Importance  of  Kxperience 

Up  to  this  point,  the  transition  from  novice  to  expert  pilot  has  been  shown  to  depend  upon:  the  type  of  knowledge 
or  knowledge  structure;  the  type  of  processing  that  is  used  to  effect  a  decision;  and,  the  fast,  accurate  retrieval 
capability  of  proper  actions.  Specifically,  high  levels  of  cognitive  processing,  or  expertise,  were  characterized  by  the 
predominant  use  of  procedural  knowledge  and  an  autonomous  processing  ability.  In  addition,  the  performance  of 
highly  competent  pilots  indicates  the  ability  to  rapidly  access  and  efficiently  utilize  their  experienced  based 
knowledge  with  marked  increase  in  processing  speed  and  accuracy,  or  appropriateness.  These  characteristics  were 
seen  to  be  based  on  an  organized,  modifiable  knowledge  structure  (schemata)  derived  from  experience,  whereas, 
novice  pilots  can  adequately  perform  using  methodical,  heuristic  thinking,  their  cognitive  performance  is  limited  by 
their  inability  to  infer  additional  knowledge  from  llic  specific  stimuli  or  cues  of  a  particular  situation. 

In  contrast  to  novices,  experienced  pilots  can  generate  inferences  in  new  situations  based  upon  the  cues  and  context 
of  the  specific  task  at  hand.  This  forward  infcrcncing  capability  is  based  upon:  the  content  of  their  aviation 
knowledge  structure;  die  procedural  organization  that  experience  has  developed;  and,  the  ability  to  go  between  the 
two  and  apply  the  proper  solution  for  the  current  problem.  The  documented  performance  of  highly  competent  pilots 
with  extensive  aviation  knowledge  bases  in  emergency  “saves”  provides  a  snapshot  of  the  powerful  problem  solving 
abilities  of  human  experts.  These  pilots  have  demonstrated  the  unique  ability  to  utilize  a  large  knowledge  base  in  an 
efficient,  automatic  manner  while  simultaneously  tailoring  their  decisions  to  the  situational  demands.  The  expert’s 
cognitive  processing  can  accomplish  this  with  minimum  reliance  on  time  consuming  search  of  declarative 
knowledge  and  heuristics  compared  to  the  less  experienced  pilot  decision  making  techniques.  Furthermore,  the  expert 
pilot  can  develop  effective  solutions  in  a  “knowledge-lean”  situation  with  ambiguous  or  contradicting  information 
and  in  the  presence  of  novel  cues  or  task  demands  never  before  experienced.  Therefore,  a  significant  focus  for 
understanding  and  training  expertise  will  require  additional  understanding  of  how  experience  influences  knowledge 
structures  that  arc  acquired  over  long  periods  of  time,  how  experts  normally  use  that  structure  and  how  that  use  can 
be  altered  to  “adapt”  to  new  problems. 

As  stated  above,  experienced  pilots  use  more  global  pattern  recognition,  retrieval  and  inferences.  These  cognitive 
processes  frec-up  working  memory  and  improve  the  pilot’s  capability  to  plan  ahead,  efficiently  monitor  his  time  and 
attention  resources,  carryout  the  normal  "housekeeping”  tasks  (i.e.,  aviate,  navigate  and  communicate),  and  still 
leave  time  for  emergency  or  unexpected  decision  making. 

These  expert  traits  arc  very  similar  to  what  is  currently  termed  "situational  awareness”  and  "cockpit  resource 
management".  Such  characteristics  and  processes  are  strived  for  in  all  pilot  training,  but  "experts"  have  the 
additional  capabilities  of  self-regulation  and  editing  or  evaluating  the  results  of  decision  making.  These  traits 
become  generalized  cognitive  processes  after  pilots  use  them  repeatedly. 

Experienced  pilots  arc  highly  procedural  and  goal  oriented.  As  a  pilot’s  information  processing  skills  become  more 
and  more  automatic  rather  than  a  controlled  cognitive  function,  they  lend  to  "experience"  a  situation  and  react  to  it 
rather  than  consciously  analyzing  and  deciding.  Because  they  have  experienced  large  meaningful 
patterns  during  daily  flying  situations,  they  are  much  better  at  relating  events  in  cause-and* 
effect  sequences  to  achieve  their  goals.  In  other  words,  experienced  pilot  cognitive  processes  are  like  expert 
processes  in  other  fields,  at  least  in  the  extremely  small  amount  of  cognitive  attention  and  conscious  processing 
required.  Finally,  these  processes  arc  fast,  as  the  example  accidents  illustrated,  and  can  be  very  effective  in  creating 
opportunistic  solutions  to  new  problems  Relationships  arc  perceived,  decisions  made  and  actions  taken  so  rapidly, 
dial  they  lake  on  the  character  of  intuition. 

Experiencing  large  global  patterns  repeatedly  throughout  an  aviation  career  enhances  a  pilot’s  cognitive  processing  by 
providing  redundancy  or  reinforcement  of  past  similar  situations,  providing  more  associative  paths  to  speed-up  recall 
in  new  situations,  and  by  providing  elaborations  or  additional  retrieval  paths  which  can  be  used  for  both  recall  and 
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inference.  However,  experience  is  mere  than  developing  and  storing  cognitive  knowledge  in  context.  As  a  pilot 
faces  each  flight  situation,  he  adopts  an  attitude  toward  it  based  upon  a  multitude  of  external  and  internal  “states”. 
This  reaction  or  psychophysiological  attitude  includes  kinesthetic,  affective  and  cognitive 
components  which  comprise  the  “experience”,  form  the  basis  of  the  experts  “global  synthetic 
knowledge”  and  provide  the  context  and  meaning  of  the  situation  to  be  used  as  a  “mental 
hook"  when  needed  for  later  decision  making  or  problem  solving. 

Experience  can  also  interfere  with  the  perception  of  a  situation  and  provide  negative  reinforcement  for  later  use  of  bad 
decision  making.  This  is  the  case  for  some  of  the  classic  aviation  accident  cause/factors  such  as:  “ducking  under” 
Decision  Height  or  M.C'mum  Descent  Altitude;  fuel  starvation/mismanagement;  inadvertent  IMC;  etc.  In  many  of 
the  accidents  attributable  to  these  causes,  the  pilot  or  crew  had  repeatedly  “gotten-away-with”  bad  decisions  and 
consequently  formed  them  into  a  bad  behavior  pattern.  Past  experience  can  also  interfere  with  the  perception  of  a 
situation  through  job  or  personal  stress,  anxiety,  fixation,  emotional  blocking,  etc.  so  that  the  synthetic  knowledge 
which  is  stored  will  not  be  representative  of  the  situation. 

Therefore,  past  experience  that  is  intended  to  be  used  for  expert  pilot  cognitive  processing 
development  must  insure  perception  of  the  essential  psychophysiological  elements  of  the 
problem  so  that  this  experience  can  be  brought  to  bear  in  a  manner  appropriate  to  successful 
problem  solving  in  new  situations.  In  particular  the  importance  of  “conditioning”  i.e.,  associating  these 
global  patterns  of  experience  with  specific  responses  appropriate  to  new  situations  has  been  shown  by  modem 
“Activity  Based  Learning”  techniques.  These  techniques  stress  kinesthetic  experiences  with  structural  apparatus 
associating  ideas  with  experiences  that  can  later  be  recalled  and  used  “intuitively”  Once  again,  the  appropriateness  of 
the  synthetic  experience  will  be  critical  to  the  subjective  associations  between  the  elements  of  the  situation  and  the 
schema/global  pattern  used  to  hold  the  solution  in  mind  for  later  use. 
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ENHANCED  TRAINING  TO  REDUCE  PILOT  ERROR  ACCIDENTS 
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INTERACTIVE  METHODS  AND  TOOLS 
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ADAPTABILITY  &  CREATIVITY 
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Dr.  Gary  A.  Klein 


Qdel  Scientist 
Klein  Associates,  Inc. 


A  COGNITIVE  MODEL  FOR  TRAINING 
DECISION  MAKING  IN  AIRCREWS 


Conclusion: 

The  Recognition  Primed  Decision  model  explains  how  people 
make  decisions  without  having  to  compare  options,  not  as  an 
alternative  to  analytical  decision  making,  but  as  one  end  of  a 
continuum  bounded  by  analytical  decision  strategies  and 
recognition  primed  decisions. 
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A  Cognitive  Model  for 
Training  Decision  Making 
in  Aircrews 


Gary  Klein 

Klein  Associates  Inc. 

582  E.  Day  ton- Yellow  Springs  Rd. 
Fairborn,  OH  45324 


Aeronautical  Decision  Making 
Workshop 
Denver,  CO 


KLEIN  AtsodalM  Inc. 


6  May  1992 


Prescriptions  for  Effective 
Decision  Making 
(Janis  &  Mann,  1977) 


Thoroughly  canvas  wide  range  of  COAs 

Survey  full  range  of  objectives 

Carefully  weigh  costs,  risks,  and 
benefits  of  each  COA 

Intensively  search  for  new  information 
for  evaluating  COAs 

Assimilate  all  new  information 

Re-examine  positive  and  negative 
consequences  of  each  COA 

Carefully  plan  to  include 
contingencies  if  various  risks 
occur 
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Serial  vs.  Concurrent  Deliberation 


Models  of  Option  Evaluation 
(A)  Concurrent  Evaluation  -  Vertical  Model 


Options 

Evaluation  Dimensions 

□ 

□ 

n 

□ 

(A) 

A  A 

A  A 

(B) 

(C) 

t  * 

t  t 

(B)  Serial  Evaluation  -  Horizontal  Model 


Options 


Evaluation  Dimensions 

□  □  □  □ 


(A) 

(B) 

(C) 
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A.  SIMPLE  MATCH  B.  DEVELOPING  A  COURSE  OF  ACTION  C.  COMPLEX  RPD  STRATEGY 


Recognition-Primed  Decision  Model 


Recognition-Primed  Decision 
(RPD)  Model 


•  Explains  how  people  can  make 
decisions  without  having  to 
compare  options 

•  Fuses  two  processes  — 

situation  assessment 
mental  simulation 


•  Asserts  that  people  use  situation 
assessment  to  generate  plausible 
COA  and  use  mental  simulation 
to  evaluate  COA 


KLEIN  Associates  Inc. 
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Key  Features  of  RPD  Model  ' 

1.  First  option  is  usually  workable 
Not  random  generation  and  selective 
retention 


2.  Serial  generation/evaluation  of  options 
Not  concurrent  evaluation 


3.  Satisficing 
Not  optimizing 

4.  Evaluation  through  mental  simulation 
Not  MAUA,  Decision  Analysis, 

or  Bayesian  statistics 


Focus  on  elaborating  and  improving 
options 

Not  choosing  between  options 

Focus  on  situation  assessment 
Not  decision  events 

DM  primed  to  act 

Not  waiting  to  complete  the  analyses 

_ _ _ KLEIN  Associate*  Inc. - 
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Factors  Affecting  the  Use  of  Recognitional  and 

Analytical  Decisions 
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Optimization  Decreased 

Combinatorial  Problem  Decreased 


Team  Research  and  Observations 


Ft.  Hood 

Battalions 

Ft.  Stewart 

Battalions  &  Brigades 

Ft.  Irwin 

Battalions  &  Brigades 

Ft.  Knox 

Platoons 

National 

Echelons  above  Corps 

Defense 

University 

Ft.  Leavenworth 

Divisions  &  Corps 

Central  Training 

Emergency  Operations  Teams 

Academy 

National  Forest 

Incident  Command  Teams 

Service 

U.S.  Navy 

Aegis  Combat  Information  Center 

Hurlburt  Field 

Blue  Flag 

Ft.  Campbell 

Helicopter  Teams 

NAS  A/ Ames 

Airline  Crews 

Industrial 

Echelons  above  Corps 

College  of  the 

Armed  Forces 

Air  Force 

Management  Teams 

Institute  of 
Technology 


137 


Aspects  of  Teamwork 

•  Coordination 

•  Leadership 

•  Adaptation 

•  Assertiveness 

•  Communication 

•  Flexibility 

•  Decision  Making 

•  Cooperation 

•  Morale 

•  Shared  Mental  Model 

•  Anticipation 

•  Conflict  Management 

•  Stress  Management 

•  Work  Distribution 

•  Information  Exchange 

•  Clarification 

•  Team  Reinforcement 

•  Interactions 

•  Supportiveness 

•  Cohesiveness 

•  Power  Distribution 

•  etc. 

-  KLEIN  AsmcMm  ln& 
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Team  Mind 
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Understanding  the  team  mind. 


Cognitive  Process  of  Teams 


•  Selective  attention 

•  Metacognition 


•  Memory  (limited-capacity 
working  memory;  retrieval 
from  long-term  memory) 

•  Intentional  behavior 


•  Mental  simulation 

•  Reaction  time 


KLEIN  Associates  Inc. 
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Key  Features  &  Critical  Processes  of  Decision  Making  Teams 
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Myths  about  Team  Decision  Making 


Exercises  =  Training 

Instructors  can  -*  feedback 

Team  Decision  Training  is  expensive 

Team  Decision  Training  must  be 
streamlined 
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Recommendations  for 
Team  Decision  Training 


Model-based  approach 
Detect  strengths 
Detect  weaknesses 
Detect  missing  behaviors 

Immediate  feedback 

Feedback  around  specific  behaviors 
Generic  feedback  is  less  useful 

On-line  coaching 

Incorporate  Team  Decision  Training 
into  exercises 

Design  exercises  for  Team  Decision 
Training 

V  J 


Or.  John  R.  Bloomfield 


Research  Psychologist 
Honeywell,  SRC 


ELEMENTS  OF  A  THEORY  OF 


Conclusion: 

The  preliminary  theory  provides  a  framework  that  can 
incorporate  a  broad  range  of  decisions  and  decision  situations: 
ranging  from  the  tightly-defined  situations  investigated  by 
classical  decision  theorists  all  the  way  to  complex  real-world 
situations,  involving  proficient  decision-makers,  under  a  great 
deal  of  stress,  and/or  time  pressure. 


145 


T 


ELEMENTS  OF  A  THEORY  OF  NATURAL  DECISION  MAKING 


John  R.  Bloomfield 

Honeywell  Systems  and  Research  Center  (MN65-2500) 
3660  Technology  Drive 
Minneapolis,  Minnesota  55418 
U.S.A. 


Introduction 

The  preliminary  decision  theory  presented  here  is  in  the  process  of  being  developed  from  the 
decision  characterization  framework  presented  to  the  Ergonomics  Society  in  1990  at  the  Leeds 
meeting  (Bloomfield,  1990).  It  should  perhaps  be  described  as  a  theoretical  framework. 

There  are  many  models  and  theories  of  decision  making.  Why  do  we  need  another  one?  Many 
theories  are  focussed  narrowly  on  choice.  Prescriptive  theories,  that  deal  with  how  choices 
should  be  made,  suggest  quite  different  mechanisms  underlie  choice  than  are  suggested  by 
descriptive  theories,  that  are  concerned  with  how  choices  are,  in  fact,  made.  And,  the  views  of 
theorists  who  have  conducted  decision-making  experiments  in  the  laboratory  often  appear  to  be 
in  conflict  with  the  views  of  theorists  who  have  observed  decision  makers  in  real-life  stuations. 
However,  when  examined  closely,  it  can  be  seen  that  these  theories  should  not  be  directly 
compared:  they  deal  with  different  types  of  decision,  use  different  decision  paradigms,  and/or 
use  decision  makers  of  different  skill  levels. 

The  preliminary  theory  provides  a  framework  that  can  incorporate  a  broad  range  of  decisions 
and  decision  situations:  ranging  from  the  tightly-defined  situations  investigated  by  classical 
decision  theorists,  in  which  naive  decision  makers  perform  unfamiliar,  highly-structured, 
context-limited,  decision-making  tasks,  often  without  real-world  constraints,  all  the  way  to 
complex  real-world  situations,  involving  proficient  decision-makers,  under  a  great  deal  of 
stress,  and/or  time  pressure. 

In  the  sections  that  follow,  the  elements  of  this  theory  are  discussed.  The  theory  takes  a  broad 
view  of  decision  making,  and  includes  some  elements  that  are  omitted  in  most  other  formal 
decision  theories. 


Types  of  Decision 

Four  types  of  decision  can  be  isolated,  mainly  on  the  basis  of  their  importance  to  the  decision 
maker. 

Consequential  Decisions — The  term  "consequential  decisions"  was  used  by  Janis  and 
Mann  (1977)  to  describe  crucial  decisions,  that  could  have  a  great  impact  on  the  decision 
maker's  life,  such  as  decisions  about  whether  to  get  married  or  divorced;  whether  to  buy  a  new 
house  or  to  have  major  surgery;  whether  to  attack  an  approaching  enemy  aircraft;  or  whether  to 
take  off  or  abort  when  an  aircraft  faces  adverse  conditions. 
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Moderately  Important  Decisions — Moderately  important  decisions  are  out  of  the  ordinary 
decisions  about  such  things  as  whether  to  buy  a  compact  disc  player,  or  what  to  do  on  your  next 
vacation. 


Everyday  Decisions — Examples  of  everyday  decisions  are:  deciding  what  time  to  set  the 
alarm  clock  for  next  morning,  what  to  have  for  lunch,  or  whether  to  phone  a  colleague  today. 

Subordinate  Decisions — A  chain  of  subordinate  decisions  may  be  necessary:  sometimes 
before  a  decision  can  be  made  (for  example,  in  order  to  make  a  decision  about  what  to  do  on 
vacation  this  year,  you  may  have  to  decide  when  you  can  take  time  off  from  work,  and  how 
many  days  you  can  take,  what  you  can  afford  to  spend  on  a  vacation,  and  whether  or  not  you  to 
take  your  children  with  you)  and  sometimes  as  a  result  of  a  decision  being  made  (for  example, 
after  you  make  the  decision  to  look  for  a  new  house,  you  may  have  to  decide  in  which  area  of 
town  should  you  search  for  a  new  house,  what  attributes  the  new  house  should  have,  and 
which  estate  agent  you  should  employ  to  sell  your  current  house). 


The  Decision  Making  Process 


Figure  1:  Model  of  the  Decision  Process 


Wickens'  (1991)  model  of  the  human  infomation-processing  system  provided  the  basis  for  the 
preliminary  model  of  the  decision  making  process.  An  overview  of  the  preliminary  model  is 
shown  in  Figure  1 .  As  the  figure  shows,  the  process  is  influenced  by  the  abilities,  experience, 
level  of  attention,  and  objectives  of  the  decision  maker.  The  decision  making  process  has  four 
main  elements  or  stages.  In  the  first,  information  is  acquired  from  the  world.  This  information 
is  passed  on  to  the  second  stage,  where  the  decision  maker  has  to  determine  the  nature  of  the 
problem.  In  the  third  stage,  he/she  selects  a  course  of  action.  Then,  in  the  fouth  stage,  this 
course  of  action  is  implemented. 
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The  overview  does  not  show  the  iterative  loops  in  the  model.  These  will  be  pointed  out  as  we 
examine  in  more  detail  the  various  elements  or  stages  of  the  model. 


The  Decision  Maker 

The  decision  maker's  level  of  expertise  will  have  a  major  effect  on  the  decision  making  process. 
Each  person's  place  on  the  continuum  from  novice  to  expert  decision  maker  varies,  to  some 
extent,  with  the  particular  decision  situation  that  he/she  is  facing:  an  expert  in  one  situation,  may 
be  only  a  competent  decision  maker  in  a  second  situation,  and  a  novice  in  a  third.  The  decision 
maker's  degree  of  expertise  is  a  function  of  his/her  ability  and  knowledge  and  experience.  The 
quality  of  his/her  decisions  will  also  be  affected  by  his/her  level  of  attention  and  objectives. 

Abilities — A  number  of  abilities  are  of  importance  in  carrying  out  decisions.  They  include  the 
ability  to  acquire  information,  to  process  information,  to  interpret  information,  to  assess 
decision  situations,  to  select  an  approach  to  making  a  decision,  to  generate  alternatives,  to 
evaluate  alternatives,  to  select  among  alternatives,  and  to  implement  alternatives.  This  list  of 
abilities  borrows  some  of  the  concepts  suggested  by  Sternberg  (1998)  as  knowledge-acquisition 
components,  and  metacomponents  in  his  triarchical  theory  of  intelligence. 

Experience — Each  individual's  decision  making  experience  will  vary  within  a  range.  He/she 
may  have  previously  encountered  a  specific  decision  situation,  or  situations  similar  to  the 
current  situation  he/she  faces,  or  decision  situations  in  general,  and  he/she  may  have 
encountered  them  very  frequently,  quite  often,  sometimes,  or  very  infrequently. 

Attention — The  extent  to  which  a  decision  maker  is  able  to  attend  to  a  particular  decision,  may 
affect  the  speed  with  which  the  decision  making  process  is  completed  and/or  the  appropriateness 
of  the  outcome  selected.  When  there  are  many  competing  sources  of  stimulation,  and  perhaps 
several  decisions  that  simultaneously  must  be  made,  the  allocation  of  sufficient  attentional 
resources  is  difficult. 

Objectives — The  decision  maker's  objectives  come  from  externally-imposed  instructions 
and/or  internally-derived  desires;  they  may  be  explicit  or  implicit;  and  they  may  be  clearly- 
defined,  ambiguous,  or  even  contradictory.  Particular  combinations  of  these  qualities  of  the 
objectives  may  differentially  affect  the  decision  making  process. 


Information  Acquisition 

Figure  2  shows  how  incoming  data  is  acquired  from  the  world  by  the  senses,  then  passes  on  to 
be  processed.  The  decision  maker  then  interprets  the  resultant  information.  After  this,  if  the 
decision  maker  does  not  have  a  clear  picture  of  the  situation,  more  data  processing  may  be 
required,  or  he/she  may  have  to  go  back  further  in  the  chain  to  acquire  more  data.  On  the  other 
hand,  if  a  clear  enough  picture  has  emerged,  the  information  developed  in  this  stage  moves  on 
to  the  Problem  Recognition  stage. 
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Figure  2:  The  Information  Acquisition  Stage 


Problem  Recognition 

As  Figure  3  shows,  at  the  beginning  of  the  Problem  Recognition  stage,  the  decision  maker  must 
determine  whether  a  decision  is  required.  There  are  three  possible  responses  to  this  question. 
First,  if  a  decision  is  not  required,  die  incoming  information  still  has  to  be  processed  and  may 
require  a  response,  but  the  way  in  which  all  this  is  achieved  is  outside  the  concerns  of  the 
decision  model.  Second,  if  the  determination  as  to  whether  a  decision  is  required  cannot  be 
made,  because  the  available  information  is  insufficient,  the  decision  maker  will  need  to  return  to 
the  Information  Acquisition  stage.  And  third,  if  a  decision  is  required,  the  next  step  for  the 
decision  maker  is  to  assess  the  decision  situation. 

Bloomfield  (1990)  suggested  that  the  decision  maker  assesses  the  situation  in  terms  of  a  number 
of  attributes.  If  the  he/she  is  proficient,  the  assessment  is  likely  to  be  carried  out  in  a 
comprehensive,  accurate,  and  efficient  way.  On  the  other  hand,  if  he/she  is  a  novice,  it  may  be 
only  partially  carried  out,  and  it  may  be  done  inadequately,  and/or  inefficiently. 

The  attribute  suggested  by  Bloomfield  are  as  follows: 

Decision  Type — Is  the  choice  to  be  made  one  of  selecting  among  alternatives,  or  one  of 
selecting  the  time  and/or  the  location  in  which  to  initiate  an  already-chosen  course  of  action? 

Familiarity — Is  the  a  situation  a  familiar  one,  either  because  the  decision  maker  has 
encountered  it  frequently  before,  or  because  he/she  can  match  it  to  expectations  derived  from 
training,  briefing  or  some  other  form  of  prior  knowledge? 
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Figure  3:  The  Problem  Recognition  Stage 


Static  or  Dynamic — Is  the  decision  to  be  made  in  a  situation  which  is  static  —  so  that  the 
conditions  do  not  change  as  a  function  of  time  —  or  in  a  situation  which  is  changing 
dynamically? 

Time-Relaxed  or  Time-Pressured — Is  the  decision  to  be  made  under  time-relaxed  or  timc- 
pressured  conditions? 

Degree  of  Ambiguity — Is  the  situation  one  that  is  well-defined,  or  is  it  ambiguous  and 
difficult  to  interpret? 

Information  Rate — Is  there  a  high  or  low  rate  of  incoming  information? 


150 


Interference — Is  there  interference  from  any  source,  or  sources,  of  irrei  ivui.t  stimulation? 

After  assessing  the  situation,  the  decision  maker  determines  whether  more  information  is 
required.  If  it  is  required,  he/she  must  determine  whether  there  time  to  acquire  this  information? 
If  there  is  time,  then  he/she  moves  back  to  the  Information  Acquisition  stage;  on  the  other  hand, 
if  there  is  insufficient  time,  then  he/she  moves  to  the  next  stage.  Option  Selection.  If  the 
decision  maker  already  has  enough  information  he/she  also  moves  on  to  the  Option  Selection 
stage. 


Option  Selection 

As  Figure  4  shows,  the  decision  maker's  first  step  in  Option  Selection  is  to  determine  whether 
any  alternative  courses  of  action  are  available.  If  the  situation  is  very  familiar,  and  the  decision 
maker  is  proficient,  the  generation  process  may  seem  effortless  as  a  number  of  possible  options 
spring  readily  to  the  decision  maker's  mind,  allowing  him/her  to  move  quickly  to  the  Option 
Evaluation  stage.  However,  if  the  situation  is  unfamiliar,  and/or  if  the  decision  maker  is  a 
novice,  it  is  less  likely  that  there  will  be  any  readily-available  options:  instead,  he/she  may  need 
to  develop  alternative  courses  of  action.  There  are  also  other  circumstances  under  which  it  may 
be  necessary  to  develop  new  options:  for  example,  even  if  the  situation  is  familiar,  there  may  be 
occasions  when  all  the  readily-available  options  prove  to  be  unacceptable,  so  that  the  decision 
maker  must  generate  new  possibilities  before  moving  on  to  the  next  step,  which  is  to  evaluate 
options. 

The  process  by  which  the  decision  maker  evaluates  options  is  likely  to  vary  with  the 
circumstances  surrounding  each  decision  that  is  made.  Many  evaluation  processes  have  been 
suggested.  Several  of  them  are  included  in  Figure  4  and  listed  below: 

Optimizing — The  prescriptive,  classical  decision  theory  approach,  stemming  from  the  work 
of  von  Neumann  and  Morgenstem  (1947)  and  Ward  Edwards  (1954,  1961),  suggests  that  a 
decision  maker  should  take  account  of  the  probabilities  of  all  possible  consequences  of  each 
alternative  that  could  be  selected,  compare  each  alternative  with  all  other  alternatives  on  all 
dimensions,  and  then  select  the  alternative  that  would  produce  the  optimal  outcome.  In  practice, 
i.e.  outside  of  the  laboratories  of  classical  decision  theorists,  it  is  hard  to  find  many  situations 
in  which  decision  makers  actually  use,  or  even  attempt  to  use,  an  optimizing  rule — two 
situations  in  which  they  may  in  fact  use  them  are  (1)  when  trying  to  select  winners  at  the  race 
track,  and  (2)  in  some  instances  in  the  domain  of  health  care.  There  are  several  problems  with 
using  an  optimizing  approach  successfully:  unless  he/she  faces  a  simple  well-defined  situation, 
the  decision  maker  does  not  have  enough  processing  capacity  to  be  able  to  compare  all  the 
options  on  all  the  dimensions;  humans  are  poor  at  assessing  probabilities;  there  may  be 
insufficient  time  to  use  an  optimizing  technique  and,  as  Miller  and  Starr  (1967)  pointed  out, 
even  when  there  is  enough  time,  the  optimal  strategy  may  not  be  to  use  an  optimizing  rule — 
sometimes  the  "optimal"  outcome  it  would  produce  would  not  be  cost  effective,  because  of  the 
high  cost  of  actually  using  such  a  rule  in  considering  and  comparing  all  the  alternatives. 

Suboptimizing — Because  of  these  problems,  Janis  and  Mann  (1977)  suggested  that  a 
decision  maker  who  tries  to  use  an  optimizing  rule  may  end  up  with  a  suboptimizing  rule 
instead,  where  he/she  maximizes  some  utilities  that  he/she  expected  to  gain  at  the  expense  of 
losing  other  utilities.  Whether  the  decision  maker  is  satisfied  or  dissatisfied  with  the  resultant 
outcome  will  depend  on  the  relative  importance  of  the  lost  utilities. 
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Figure  4:  The  Option  Selection  Stage 


Saticfidng — In  many  situations,  Simon  (1955)  suggested  that  the  decision  maker  does  not  try 
to  optimize,  but  instead  uses  a  satisficing  strategy,  in  which  he/she  considers  alternatives 
sequentially,  selecting  the  first  alternative  that  is  good  enough. 

Elimination  By  Aspects — Tversky  (1972)  suggested  another  alternative  to  optimizing:  with 
it,  the  decision  maker  essentially  uses  a  sequential  narrowing-down  process,  in  which  the 
possible  alternatives  are  compared  on  one  dimension  at  a  time.  The  decision  maker  first 
determines  which  alternatives  arc  able  to  satisfy  his/her  most  important  requirement,  discarding 
those  alternatives  that  do  not  meet  it,  then  moves  on  to  the  next  most  important  requirement. 
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For  example,  when  buying  a  car,  price  might  be  the  decision  maker’s  most  important 
consideration,  and  all  cars  that  cost  more  than,  say  $15,000,  would  be  eliminated.  Having  high 
gas  mileage  might  be  the  next  most  important  consideration,  and  those  cars  not  meeting  this 
requirement  would  be  the  next  to  be  eliminated.  This  procedure  would  continue,  with 
requirements  that  were  progressively  less  important,  until  only  one  alternative  remained. 

Recognition-primed  approach — Gary  Klein  (1989)  used  Simon's  satisficing  suggestion  in 
developing  a  detailed  account  of  proficient  decision  making.  Klein  suggested  that,  when 
considering  the  available  alternatives  sequentially,  the  proficient  decision  maker  uses  mental 
simulation  to  assess  the  likely  outcome  of  employing  each  alternative.  As  soon  as  he/she  finds 
an  alternative  that  will  lead  to  an  acceptable  outcome,  he/she  stops  the  assessment  process  and 
moves  to  the  Option  Implementation  stage.  Often,  the  decision  maker  only  needs  to  assess  one 
alternative — his/her  expertise  in  the  decision  situation  assessment  step  of  the  Problem 
Recognition  stage  enabling  him/her  to  produce,  an  alternative  that  is  acceptable  immediately. 

Evidence  to  supports  Klein's  recognition-primed  model  has  been  obtained  in  a  number  of 
studies  in  which  information  was  gathered  from  expert  decision  makers,  including  fire  ground 
commanders  (Klein,  Calderwood,  and  Clinton-Cirocco,  1986),  tank  platoon  leaders  (Brezovic, 
Klein,  and  Thordsen,  1987),  combat  pilots  (Bloomfield,  Peio,  Lehman,  Masters,  and 
Boettcher,  1989),  and  business  executives  (Sherwood- Jones,  1989). 

When  a  decision  maker  faces  an  unfamiliar  situation,  in  which  he/she  does  not  know  how  to 
produce  an  acceptable  outcome,  he/she  is  likely  to  attempt  to  use  a  criterion  of  optimality,  since 
selection  of  the  optimal  course  of  action  should  guarantee  an  acceptable  outcome,  unless,  of 
course,  no  solution  would  have  been  acceptable. 


Option  Implementation 


Figure  5.  The  Option  Implementation  Stage 


Figure  5  shows  the  final  decision  making  stage.  Option  Implementation.  It  always  involves  the 
first  of  the  two  steps  shown  in  the  figure,  and  often  requires  the  second  as  well. 
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Option  Initiation — Once  it  is  selected,  the  decision  maker  must  initiate  an  option,  either 
manually  or  vocally. 

Option  Monitoring  and  Adjustment — In  some  cases,  the  decision  maker  needs  to  monitor 
the  selected  course  of  action,  and  to  make  appropriate  adjustments  to  ensure  that  it  is 
implemented  properly:  for  example,  after  carrying  out  the  decision  to  fire  a  weapon,  an  operator 
may  need  to  monitor  and  adjust  the  flight  path  of  the  munition  until  it  locks  on  to  its  target 

Connolly  and  Wagner  (1988)  suggest  an  alternative  form  of  option  monitoring  and  adjustment, 
with  the  decision  maker  making  an  incremental  commitment  and  testing  a  choice  by 
experiencing  its  effects  in  a  limited  and  non-binding  way:  for  example,  by  taking  out  a  trial 
subscription  for  a  journal. 

A  second  incremental  approach  to  decision  making,  identified  by  Lindblom  and  his  co-workers 
(e.g.  Braybrooke  and  Lindblom,  1963)  makes  use  of  monitoring  and  adjustment  in  a  different 
way:  that  is,  to  gain  acceptance  of  other  people.  In  this  case,  the  decision  maker,  implements 
the  selected  course  of  action  by  making  a  series  of  adjustments,  that  he/she  hopes  are  each  small 
enough  to  be  acceptable  to  others.  With  this  approach,  which  is  sometimes  used  by  politicians, 
the  decision  maker  may  implement  a  series  of  small  adjustments  that  all  go  in  a  consistent 
direction,  eventually  producing  a  substantial  change. 


Future  Directions 

At  present,  the  preliminary  model  outlined  above  deals  with  a  range  of  diverse  decision 
situations,  from  the  highly-structured,  unfamiliar  laboratory  decision  task  faced  by  a  novice 
decision  maker,  to  the  dangerous  and  stressful  situation  faced  by  a  proficient  decision  maker.  It 
also  includes  timing  decisions  in  dynamic  situations,  where  the  decision  maker  has  already 
selected  a  course  of  action,  but  has  to  decide  when  the  conditions  are  favorable  for 
implementation,  and  it  suggests  how  different  mechanisms  of  selecting  alternatives  might  be 
used  in  diverse  decision  situations. 

The  next  step  in  developing  this  model  is  to  determine  whether  four  other  theoretical 
formulations  can  be  incorporated  into  it.  The  first  of  these  is  Montgomery's  (1983,1989)  idea 
that  decision  making  involves  the  search  for  a  dominance  structure,  with  the  decision  maker 
moving  backwards  and  forwards  between  possible  alternatives  and  desired  attributes — at  the 
very  least  more  feedback  loops  will  be  needed  in  the  model,  if  this  idea  is  to  be  incorporated. 
The  second  is  Janis  and  Mann's  (1977)  conflict-theory  decision  making  model,  which  suggests 
how  differences  in  the  circumstances  might  lead  a  decision  maker  to  react  to  a  threatening 
situation,  sometimes  successfully,  sometimes  not,  in  a  varie  of  ways,  including  ignoring  it, 
responding  to  it  rapidly  (and,  maybe,  inappropriately),  attempting  to  avoid  it,  panicking,  or 
evaluating  it  carefully.  The  third  is  Rasmussen's  (1983)  ruled-based,  knowledge-based,  skill- 
based  trichotomous  account  of  performance.  And  the  fourth  is  Pennington  and  Hastie’s  (in 
press)  account  of  explanation-based  decision  making. 
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TAXONOMY  OF  FLIGHT  VARIABLES 


Conclusion: 

The  taxonomy  developed  for  Controlled  Flight  Into  Terrain 
accidents  and  incidents  will  also  be  of  use  in  considering  other 
air  accidents  and  as  a  strawman  taxonomy  of  flight  operations 
in  general. 
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TAXONOMY  OF  FLIGHT  VARIABLES 


John  R.  Bloomfield*  ,  Lee  Levitan*,  Barry  Cooper +, 
Elizabeth  A.  Lyall+,  and  Eleana  Edens* 


Introduction 

Under  FA  A  Contract  #DTFA01-91-C-00040,  we  are  currently  investigating  Controlled 
Flight  Into  Terrain  (CFIT)  accidents  and  Controlled  Flight  Toward  Terrain  (CFTT) 
incidents,  particularly  those  that  occur  when  the  flight  crew  believe  that  the  autopilot,  flight 
director,  and/or  flight  management  system  is  in  use. 

Our  first  step  was  to  develop  a  taxonomy  of  flight  variables.  Currendy,  we  are  using  this 
taxonomy  to  review  CFIT  accidents  and  CFTT  incidents. 

We  believe  the  taxonomy  will  also  be  of  use  in  considering  other  air  accidents,  and  that  it 
will  be  a  useful  strawman  taxonomy  of  flight  operations  in  general.  The  taxonomy  has 
seven  major  categories.  They  are: 

1.  Flight  Variables 

2.  Aircraft  Equipment  Variables 

3.  Visibility  Variables 

4.  Pilot  Variables 

5.  Air  Traffic  Control  Variables 

6.  Airport  Variables 

7.  Airline  Procedures  Variables 

A  detailed  listing  of  the  flight  variables  organized  into  these  categories  is  provided  below. 
We  would  welcome  any  comments  on  items  that  should  be  modified,  or  on  any  variables 
that  have  been  omitted,  or  any  suggestions  of  how  the  variables  might  be  better  organized. 


*  Honeywell  SRC,  3660  Technology  Dr.,  Minneapolis,  MN  55418 
+  America  West  Airlines,  2323  W.  14th  St.,  Suite  309,  Tempe,  AZ  85281 

Federal  Aviation  Administration,  800  Independence  Ave.  S.W.,  Washington,  DC 
20591 
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Taxonomy  Of  Flight  Variables 
1.  Flight  Variables — 

•  Length  -short  haul  (under  45  min)/ 

medium  haul  (45  min  to  3  hr)/ 
long  haul  (over  3  hr) 

•  Schedule  -late/ontime  departure 

-eariy/ontime/late  ETA 

•  Direction  of  flight  -east/west  bound 

•  Local  time  of  day 

•  Local  time  of  year 

•  Other  segments  scheduled 

-before/after  _ 

segment  in  which  CFIT/CFTT  occurs 


2.  Aircraft  Equipment  Variables — 

•  Control  display  unit  -installed/not  installed 

-used/not  used 

-programmed  correctly/incorrecdy 

•  Map  mode  -installed/not  installed 

-used/not  used 

•  Landing  lights  -on/off 

•  Localizer/glideslope  -connected/not  connected 

•  Hard  copy  charts  -brand 

-quality 

-location  in  flightdeck 
-current  chart  used/not  used 
-correct  chart  used/not  used 
-used  by  both/one/neither  pilot 

•  Radar  altimeter  data  -available/not  available 

•  Barometric  altimeter  type 

•  Weather  radar  -in  use/not  in  use 

•  Mode/automation  level  -understood/not  understood 

•  Electronic  Flight  Instrument  System 

-installed/not  installed 
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•  Flight  director 

•  Autopilot 

•  Windscreen  wipers 
-  Approach  mode 


-used/not  used 

-used/not  used 

-in  use/not  in  use 

-vertical  speed/glideslope/other 
-engaged  correctly/incorrectly 
-understood/not  understood 


•  Navaid  -which  type 

-engaged/not  engaged 
-programmed  correctly/in  correctly 
-pilot  knew/did  not  know 
which  navaid  was  in  use 


•  Autothrottle  -installed/not  installed 

-in  use/not  in  use 


•  Ground  Proximity  Warning  System 

-installed/not  installed 
-active/inhibited 

-sounded  waming/did  not  sound  warning 
-warning  heeded/waming  not  heeded 

•  Aircraft  configuration  at  CF1T  site 

•  Minimum  Equipment  List  items 

•  Malfunctions  while  in  flight 


3.  Visibility  Variables — 

•  Visibility  -at  CFIT/CFTT  site 

-reported  at  destination 

•  Sun  -crew  looking  into/not  looking  into 

•  Time  of  day  -day/dusk/night 

•  Meteorological  conditions 

-enroute 

-at  destination 

-at  CFIT/CFTT  site 


4.  Pilot  Variables — 

(These  apply  to  both  the  pilot  flying  and  the  pilot  not-flying) 

•  Age 

•  Experience  -total  flying  hours 

-flying  hours  with  particular  aircraft 
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-experience  with  destination  airport 
-experience  with  precision  approaches 
-experience  with  nonprecision  approaches 
-experience  with  hilly/mountainous  terrain 
-experience  with  flat  terrain 


Seat  position 

Degree  of  fatigue  -low/medium/high 

Amount  of  stress  -low/medium/high 

Medical  issues 

Crew  member  familiarity  with  each  other 

Intercrew  communications 

-poor/good 

Interpersonal  issues  between  crew  members 

-good/neutral/bad 

Distractions 


5.  Air  Traffic  Control  Variables- 


Crew-expected  and  actual  ATC  instructions 

-matched/did  not  match 

Controller  attention  to  involved  aircraft 

-yes/no 

Vectoring  -none/some/much 

-early/late 

Speed  change  requests  to  aircrew 

-no  request/increase/hold  /decrease  speed 

Controller  workload  -low/medium/high 

Controller  degree  of  fatigue 

-low/medium/high 

Interpersonal  issues  between  ATC  and  flight  crew 

-existed/did  not  exist 
-nature  of  issues 


Communication  misunderstandings  between  ATC  and  flight  crew 

-existed/chd  not  exist 
-nature  of  misunderstanding 
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•  Did  ATC  detect  some  problem  with  the  involved  aircraft? 

-no 

/yes,  but  not  reported  to  crew 
/yes,  and  was  reported  to  crew 
-content  of  report 

•  Minimum  Safe  Altitude  Warning 

-installed/not  installed 

-operating/not  operating 

-did/did  not  sound  warning  to  ATC 

-ATC  did/did  not  issue  warning  to  flight  crew 

•  Altitude  clearance  instructions 

-nature  of  instructions 
-when  issued 

•  Aircraft  cleared  for  approach 

-no/yes 

-if  yes  at  what  point  in  flight 
-type  of  approach 

•  Communications  frequency 

-congested/not  congested 
-blocked/not  blocked 

•  Controller  experience  -low/medium/high 

•  ATC  anomalies 


6.  Airport  Variables — 


•  Physical  features 

•  Lighting  types 


•  Lighting 


-runway  length  and  width 

-runway  lights 
-approach  fights 
-approach  slope  indicator 
/no  approach  slope  indicator 
-airport  environment  lights 
-nearby  lighting 
-freeway,  other 

-visible/not  visible  to  crew 
-used/not  used  for  vertical  guidance 


•  Navigation  aids  -type 

-location 

-quality 

-Radio  Magnetic  Indicator  indicating  navaid 
/RMI  not  indicating  navaid 
-pilot  did/did  not  select  correct  navaid 
-pilot  confused/not  confused  about  navaid 
-navaid  does/does  not 
give  distance  information 
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•  Approach  -visual/nonprecision/precision 

•  Terrain  at  and  just  preceding  CFIT/CFTT  site 

-hilly/mountainous/flat/water 


7.  Airline  Procedures  Variables — 

To  date,  we  have  not  developed  this  category.  Airline  procedures  may  play  a  role  in  some 
CFIT  accidents  and  CFTT  incidents.  As  we  review  the  CFIT/CFTT  literature,  we  will 
record  any  indications  that  these  procedures  did  play  a  role. 


Flight  Himm  Factors  Branch 
NASA- Ames  Research  Center 


SHARED  MENTAL  MODELS  AND 


Conclusion: 

The  model  implies  that  effective  crew  decision  making  training 
would  include  different  loci  such  as  Situational  Awareness, 
Planning,  Communication,  Resource  Assessment  and 
Prioritizing.  This  research  also  raises  the  research  issues:  What 
can  be  trained?  and  How? 
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SHARED  MENTAL  MODELS  AND 


CREW  DECISION  MAKING 


Judith  Orasanu 

NASA-Ames  Research  Center 

Mail  Stop  262-4 
Moffett  Field,  CA  94035-1000 
(415)  604-3404 

jorasanu@eos.arc.nasa.gov 


FAA/AAC  WORKSHOP  ON 
AERONAUTICAL  DECISION  MAKING 

Denver,  CO 
May  5-8,  1992 
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OUTLINE  FOR  PRESENTATION  AT  FAA/AAC  WORKSHOP  ON 
AERONAUTICAL  DECISION  MAKING,  Denver,  May  5-8,  1992 

SHARED  MENTAL  MODELS  AND  CREW  DECISION  MAKING 

Judith  Orasanu 
NASA-Ames  Research  Center 


I.  Structure  of  decisions  tasks  in  the  cockpit  and  their  coanitive  requirements 

m 


A. 


-Based  Decisions 
Go/No  Go  Decisions 
Recognition-primed  Decisions 


B.  Knowledge-based  decisions  -  WELL-structured  problems 
Choice  Decisions 
Scheduling  Decisions 


C.  Knowledge-based  decisions  -  ILL-Structured  problems 
Procedural  Management  Decisions 
Creative  Problem  Solving 


RESEARCH  ISSUES:  Is  this  the  right  taxonomy? 

What  kinds  of  decision  tasks  are  found  in  the  cockpit? 
What  cogm'ivp  *  ocesses  are  required  for  each? 

What  are  Uw  .reak  links? 


II.  What  is  known  about  effective  crew  decision  making? 

A.  Criteria  for  evaluating  crew  decisions? 

Outcomes  vs.  Performance 

B.  Process  variables  that  affect  performance 

Good  situation  awareness 
Planfulness 

Communicate  to  build  shared  mental  models 
Resource  management 

RESEARCH  ISSUES:  What  are  appropriate  CRITERIA  for  evaluating  aero  DM? 
Are  normative  criteria  relevant?  For  which  types  of  decisions? 


III.  Implications  for  training  ADM 

A.  Implications  from  analysis  of  decision  types 

B.  Implications  from  4  process  factors 
RESEARCH  ISSUES:  What  should/can  be  trained?  HOW? 
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OVERVIEW 


1.  What  is  the  structure  of  decision  tasks  in 
the  aeronautical  environment? 

2.  What  is  known  about  effective  crew 
decision  making? 

3.  What  are  the  criteria  for  judging 
aeronautical  decisions? 

4.  What  are  the  implications  of  1.  and  2.  for 
training  ADM? 
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WHAT  IS  THE  STRUCTURE  OF  DECISION  TASKS 
IN  THE  AERONAUTICAL  ENVIRONMENT? 

1.  £H-Based  Decisions 

~  Go-No  Go  Decisions 
—  Recognition-Primed  Decisions 

2.  Knowledge-based  Decisions 

-  WELL-STRUCTURED  PROBLEMS 

-  Response  selection  decisions 

-  Scheduling  decisions 

3.  Knowledge-based  Decisions 

-  ILL-STRUCTURED  PROBLEMS 

-  Procedural  management  decisions 

-  Creative  problem  solving 
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RESEARCH  ISSUES: 


1.  Is  this  the  right  taxonomy? 

2.  What  cognitive  processes  are  required  by 
each  type  of  decision? 

3.  What  are  their  weak  links? 
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WHAT  IS  KNOWN  ABOUT  EFFECTIVE  CREW 
DECISION  MAKING? 

1.  What  CRITERIA  should  be  used  to  judge 
decision  quality? 

Outcomes  vs.  Performance 

2.  PROCESS  variables  that  affect  performance 

-  Situation  awareness 

-  Metacognitive  skills 

-  Shared  mental  models 

-  Resource  management 
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RESEARCH  ISSUES: 


1.  Are  different  criteria  appropriate  for 
different  types  of  decisions? 

2.  Are  normative  criteria  ever  relevant? 

For  which  types  of  decisions? 

3.  How  do  we  know  if  crew  decision 
performance  has  improved? 
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IMPLICATIONS  FOR  TRAINING  ADM 


1.  6  decision  types  require  different  skills: 

perceptual  pattern  matching 
situation  assessment 
risk  assessment 
--  scheduling 

option  selection  heuristics 
problem  diagnosis 
-  creative  problem  solving. 

2.  Model  of  effective  crew  decision  making 
implies  different  training  foci: 

situation  assessment 
planning 

communicating  (within  cockpit  and  with 
ground  --  ATC  and  dispatch) 
resource  assessment  and  prioritizing 
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RESEARCH  ISSUES: 


1.  What  can/should  be  trained? 

2.  HOW? 
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Chief  Safety  Engineer 
Bell  Helicopter  Textron 


EMBEDDED  ADM  REDUCES  HELICOPTER 
HUMAN  ERROR  ACCIDENTS 


Conclusion: 


Judgment  Training  (e.g.,  PDM  or  ADM)  has  more  safety 
improvement  potential  than  the  total  elimination  of  all 
airworthiness  failure  causes  (a  primary  goal  since  the  start  of 
aviation). 
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HELIPROPS 
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HUMAN  ERROR  ACCIDENTS 


BELL'S  APPROACH 


1987  SAFETY/JUDGMENT  TRAINING  STARTED 


WHAT  DID  NOT  WORK  : 

TALKING  DIRECTLY  TO  THE  SIJBJEC 


o 

CM 

CO 

LU 

CO 

Z> 

< 

O 


o 

CD 

< 

<C 

CO 


=;  lu 


O 

O  H 

2  o 

—  a 

o 


o  § 

186 


LU 

m 

O 

DC 


✓v 

/\ 

/\ 


CO 

2 

Q 

< 


PILOT'S  RESISTANCE  &  FEELING  OF 
BEING  ATTACKED. 


APPROACH  THAT  WORKS: 
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8.  PROVIDE  FORMAL  A6M  INTRODUCTION 

9.  SHOW  THAT  HELIPROPS  IS  WORKING 

10.  FUTURE  TECHNOLOGY:  PC  DECISION-MAKING  SIMULATOR 


MESSAGE  RECEIVED 
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The  basic  outline  of  this  brief: 
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Mechanics  of  a  crash 

Crash  energy  management  (energy  attenuation) 
Post  Crash  Fires  &  Crash  Resistant  Fuel  Systems 
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related  to  weather  decisions  reduced  72%. 


AVIATION  ACCIDENT  RATES 

U.S.  REGISTERED  AIRCRAFT,  NTSB  DATA 
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Fig.  4.  Known  accident  cause*. 


BELL  SAFETY  TRAINING  IS  REDUCING 
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AIMED  AT  INDIVIDUAL  PILOT  TRAINING  AROUND  W< 
AVAILABLE  ON  MODELS  206B3,  206L3,  AND  412/212 


(HELicopter  PRO/essionaf  Pilot  Safety)  program 
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Roy  G.  Fox 


SAFETY  RESPONSIBILITY  CHAIN 


AVIATION  ACCIDENT  RATES 
(1975  -  1979) 
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SINGLE  ENGINE  PROHIBITIONS  NOT 
CONSISTENT  WITH  ACCIDENT  EXPERIENCES 
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*  B  489 


USA  -  REGISTERED  HELICOPTER  ACCIDENT  RATES 
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SINGLE  PISTON  SINGLE  TURBINE  TWIN  TURBINE  BELL  206 

AIRWORTHINESS  FAILURE  ACCIDENT  RATES  FOR  UK,  USA.  AND  CANADA  IN  THE  1980  S 


TIME  TRAVEL  EQUIVALENT:  AROUND-THE-WORLD  TRIPS 
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SAFETY  IS  RISK  MANAGEMENT 
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CAN  NOT  IGNORE  EltHER  PROBABILITY 


RELATIVE  RISK  OF  SERIOUS  INJURY 
IN  GENERAL  AVIATION 
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FAA  CIVIL  HELICOPTER  CRASH  SCENARIO  STUDY 
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O’*  APPLIED  TO  OCCUPANT  IN  CRASH 
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THERE  IS  NO  20Q  CRASH 
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PERCENTAGE  OF  KNOWN  CAUSES 

NTSB/FAA  (1982 -1985) 


206  SINGLE  TURBINE 


216 


ACCIDENT 


HELICOPTER  ACCIDENT  CAUSES 

NTSB  (1982-1985) 
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FLIGHT -HOUR  EXPERIENCE  OF  PILOTS  IN  HELICOPTER  MODEL  FOR  PILOT 
ERROR  ACCIDENTS  VS.  AVERAGE  PILOT  (BASED  ON  NSTB  DATA) 
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PILOT  FLIGHT  HOURS  IN  HEIICOPTCR  MAKf 
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Table  14.  Worldwide  Bell  turbine  accident  rates  (Rates  per  100,000  flight-hours) 
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206  HUMAN  ERROR  ACCIDENT 
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206  ACCIDENTS  RATE  WITH  &  WITHOUT  ADM 
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JUDGMENT  TRAINING 


Roy  G.  Fox 


Chief  Safety  Engineer 
Bell  Helicopter  Textron 


MEASURING  RISK  IN  SINGLE-ENGINE 
AND  TWIN-ENGINE  HELICOPTERS 


Conclusion: 

The  human  error  accident  rate  for  Bell  206  pilots  receiving  ADM 
training  from  1987  to  1991  was  significantly  different  from  the 
human  error  rate  for  Bell  206  pilots  from  1982  to  1986  to  a 
significance  level  of  0.05. 
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MEASURING  RISK  IN  SINGLE  -  AND  TWIN -ENGINE  HELICOPTERS 


Roy  G.  Fox 
Chief  Safety  Engineer 

Bell  Helicopter  Texton,  Inc. ,  Fort  Worth,  Texas 


ABSTRACT 

Safety  is  the  management  of  risk.  Many  decisions 
are  made  by  businesses,  government  agencies  and 
individuals  using  their  perceptions  of  an  aircraft’s 
safety.  Public  perception  of  safety  can  deny  the  in¬ 
troduction  or  expansion  of  aviation  in  specific  areas. 
Decisions  to  buy,  use,  repair,  install  improvements, 
insure,  sell,  and  replace  aircraft  are  all  related  to 
perceived  safety.  Likewise,  governmental  restric¬ 
tions  and  rule-making  are  based  on  the  perceived 
deterioration  of  safety,  as  in  the  proposed  single¬ 
engine  helicopter  restrictions  of  ICAO  Annex  6.  Ac¬ 
curate  aircraft  safety  measurements  are  thus  essen¬ 
tial  to  bring  perceived  and  actual  safety  together. 
Such  accuracy  also  provides  realistic  corrective  ac¬ 
tions  for  safety  problems  and  evaluation  of  desirable 
and  undesirable  aspects  of  different  aircraft  configu¬ 
rations,  as  well  as  allowing  individuals  to  determine 
their  risk  in  flying  in  specific  types  of  aircraft.  Ex¬ 
isting  safety  measuring  methods  are  discussed, 
along  with  the  advantages,  disadvantages,  and  cor¬ 
rectness  of  each  method.  Recent  safety  training  and 
its  effects  are  discussed,  related  to  improved  pilot 
judgment  and  significant  reductions  in  accident 
rates- without  any  regulatory  changes. 

INTRODUCTION 

Safety  has  always  been  a  paramount  concern  in  avi¬ 
ation.  Safety  is  not  an  absolute;  rather,  it  is  a  rela¬ 
tive  measure  of  the  risk  involved  when  flying  in  an 
aircraft.  Several  methods  are  used  by  publications 
that  attempt  to  measure  safety.  Some  of  these  meth¬ 
ods  are  misleading  and  inaccurate,  and  create  the 
perception  of  a  low  level  of  safety  in  helicopters. 
Misconceptions  about  helicopter  safety  can  cause 
overly  restrictive  regulations  and  prohibit  the  use  of 
safe  aircraft.  Thus,  accurate  measuring  of  helicop¬ 
ter  safety  is  crucial  to  the  helicopter  operators  and 
the  flying  public. 
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There  is  a  continuing  question  of  whether  an  occu¬ 
pant  is  safer  in  a  single-engine  helicopter  or  a  twin- 
engine  helicopter.  Some  proponents  say  that  two  en¬ 
gines  must  be  better  than  one.  Others  say,  "We 
have  two  engines  in  commercial  fixed-wing  air¬ 
planes;  therefore,  helicopters  also  need  two  en¬ 
gines.”  However,  facts  do  not  support  application  of 
"fixed-wing  thinking”  to  helicopters.  Helicopters 
have  unique  uses  and  designs  and  are  operated  in 
difficult  environments.  Thus,  helicopters  are  differ¬ 
ent  from  fixed-wing  airplanes.  One  must  look  at  all 
causes  of  accidents  and  injuries,  not  just  at  compo¬ 
nents  like  engines  or  tail  rotor  blades.  This  paper 
addresses  the  safety  issues  for  both  single-engine 
and  twin-engine  helicopters. 

Accident  data  from  the  United  States  of  America 
(USA),  the  United  Kingdom  (UK),  and  Canada  were 
analyzed  to  determine  the  risk  to  occupants  of 
single-  and  twin-engine  powered  helicopters.  These 
three  nations  (States)  account  for  about  824b  of  all 
known  (non-Soviet  bloc)  civil  helicopters.  Although 
the  subject  of  this  paper  is  rotary  wing  aircraft,  the 
methodology  is  equally  applicable  to  fixed-wing  air¬ 
planes. 


WHY  MEASURE  SAFETY? 

Many  important  decisions  made  by  businesses,  gov¬ 
ernment  agencies,  and  individuals  are  based  on  the 
perceived  safety  of  an  aircraft.  Decisions  to  buy, 
use,  fix,  improve,  insure,  and  sell  or  replace  an  air¬ 
craft  are  related  to  perceived  safety.  Likewise,  gov¬ 
ernment  operational  prohibitions  are  based  on  the 
perceived  deterioration  of  safety.  For  example,  the 
recent  Amendment  1  to  International  Civil  Aviation 
Organization  (ICAO)  Annex  6,  Part  III  (Ref.  1)  es¬ 
tablishes  three  categories  of  helicopter  performance 
and  recommends  certain  operational  limitations. 
The  categories  are 

Performance  Class  1.  Includes  multiengine 
helicopters  that  are  capable  of  continuing  normal 
operations  with  one  engine  inoperative,  regardless 
of  when  the  engine  fails. 
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Performance  Class  2.  includes  multiengine 
helicopters  that  are  capable  of  continuing  flight 
after  one  engine  fails,  except  that  a  forced  landing 
would  be  required  following  an  engine  failure  be¬ 
tween  takeoff  and  a  specific  point  and  between  a  spe¬ 
cific  point  and  landing. 

Performance  Class  3.  Refers  to  single-engine 
helicopter  operations;  a  forced  landing  would  be  re¬ 
quired  after  an  engine  failure. 

Amendment  1  to  ICAO  Annex  6,  Part  III  recom¬ 
mends  prohibition  of  the  use  of  Performance  Class  3 
single-engine  helicopters  for  instrument  flight  rule 
(IFR)  flights,  night  flying,  flights  out  of  sight  of  the 
earth’s  surface,  flights  with  cloud  ceilings  of  less 
than  600  feet  or  visibility  less  than  1,500  meters, 
and  flights  to  elevated  structures  (heliport).  ICAO 
itself  does  not  regulate  world  standards;  however,  it 
recommends  that  individual  member  states  adopt 
its  criteria  into  their  own  regulations.  The  United 
States  and  many  other  countries  have  not  adopted 
the  recent  recommendations  of  ICAO  Amendment  1 
to  Annex  6,  Part  III. 

Since  single-engine  helicopters  account  for  three  out 
of  four  helicopters  in  the  world,  adoption  of  this 
amendment  would  have  a  drastic  effect  on  the  heli¬ 
copter  community  and  on  the  public  benefit  derived 
from  helicopter  use.  Some  single-engine  helicopter 
operations  will  no  longer  be  performed,  due  to  the 
higher  costs  involved,  if  twin-engine  helicopters  are 
mandated.  Most  multiengine  helicopter  operations 
are  conducted  in  Performance  Class  2.  Since  the  ac¬ 
cident  data  do  not  discriminate  between  perfor¬ 
mance  classes,  the  safety  comparisons  of  Perfor¬ 
mance  Classes  2  and  3  from  the  available  data  is  ac¬ 
complished  in  this  paper  by  looking  at  the  differ¬ 
ences  between  single-engine  (Performance  Class  3) 
and  multiengine  (Performance  Class  2  )  operations. 
The  performance  class  restrictions  on  helicopter  op¬ 
erations  in  accordance  with  the  ICAO  Amendment  1 
change  includes  the  prohibition  of  single-engine  he¬ 
licopter  operations  involving  transport  of  passen¬ 
gers,  cargo,  or  mail  for  remuneration  or  hire,  and 
general  aviation  uses.  This  prohibition  is  based  on  a 
perceived  belief  that  twin-engine  helicopters  are  al¬ 
ways  safer  than  single-engine  helicopters  in  all  en¬ 
vironments.  Thus,  accurate  helicopter  safety  mea¬ 
surements  are  critical  to  ensure  that  perceived  safe¬ 
ty  and  actual  safety  may  be  similar.  Such  accuracy 
also  allows  prioritized  correction  of  safety  problems 
and  the  evaluation  of  desirable  and  undesirable  as¬ 
pects  of  different  aircraft  configurations.  This  is  of 


personal  importance  to  an  individual,  allowing  a 
person  to  determine  his  risk  when  flying  in  a  specif¬ 
ic  type  of  aircraft. 

Why  Worry  about  Safety? 

Why  do  people  worry  about  safety  in  the  first  place? 
The  primary  reason  is  that  no  one  wants  to  get  hurt 
or  die.  Since  none  of  us  wants  to  think  about  our 
own  death  or  injury,  we  tend  to  tell  ourselves  "I  am 
never  going  to  be  in  an  accident,  therefore  I  won’t 
have  to  worry  about  being  injured  or  killed.”  The 
next  step  in  this  internal  stress  coping  action  is  to 
assume  that  all  that  is  needed  to  accomplish  the  goal 
is  to  prevent  all  accidents.  This  internal  protection 
mechanism  helps  each  of  us  go  through  all  of  the 
stresses  of  each  day.  Aviation  accident  prevention  is 
based  on  this  concept:  "If  I  can  prevent  the  emergen¬ 
cy,  I  won’t  have  to  worry  about  my  pain  and  my 
death.”  This  human  coping  mechanism  works  well 
for  the  average  individual;  but  management  (avi¬ 
ation  and  regulatory)  must  go  beyond  to  first  deter¬ 
mine  the  actual  risk  and  subsequently  manage  the 
risk  to  an  acceptable  level.  Safety  is  the  manage¬ 
ment  of  risk. 

HELICOPTERS  AND  AIRPLANES  RESPOND 
DIFFERENTLY  TO  POWER  LOSS 

If  a  power  loss  occurs,  the  resulting  emergency  land¬ 
ings  are  significantly  different  for  airplanes  than  for 
helicopters.  To  maintain  control  of  an  airplane,  its 
airspeed  must  stay  above  wing  stall  speeds  until 
ground  contact.  This  means  the  airplane  airspeed  at 
ground  contact  will  be  typically  60  to  100  knots. 
This  high  speed  requires  a  shallow  approach  angle 
and  a  long  cleared  landing  site.  Any  obstructions 
(e.g.,  trees,  buildings,  fences,  or  ground  irregulari¬ 
ties)  will  be  impacted  with  significant  crash  forces 
and  resulting  injuries. 

Conversely,  all  helicopters  have  a  safety  feature  ca¬ 
pability  to  make  an  unpowered,  controlled  landing, 
called  "autorotation.”  Figure  1  shows  the  airflow 
during  this  emergency  procedure.  The  pilot  controls 
the  pitch  of  the  main  rotor  blades  at  all  times.  In 
normal  flight  under  power,  the  air  is  pulled  through 
the  main  rotor  disc  and  thrust  downward,  providing 
lift  to  hold  the  helicopter  in  the  air  and  controlling 
the  aircraft.  In  unpowered  flight  or  an  emergency 
descent,  the  pilot  enters  autorotation  and  changes 
the  pitch  on  the  main  rotor  blades  to  allow  the  air  to 
come  upward  through  the  main  rotor  disc  as  the  he¬ 
licopter  is  descending.  This  airflow  turns  the  main 
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rotor  blades  like  the  wind  turns  a  windmill.  The  pi¬ 
lot  prevents  overspeeding  of  the  main  rotor  by  con¬ 
verting  some  of  this  energy  being  gained  to  lift. 
Thus,  the  spinning  main  rotor  acts  like  a  parachute 
and  a  near-constant  descent  rate  is  maintained.  The 
main  rotor  blade  structure/weights  store  this  rota¬ 
tional  energy  in  a  manner  similar  to  a  giant  fly¬ 
wheel.  Since  the  helicopter  is  fully  under  control, 
the  aircraft  can  be  maneuvered  to  the  best  landing 
site,  allowing  a  steep  approach  into  a  confined  area. 
Upon  approaching  the  selected  landing  site,  the  pi¬ 
lot  flares  (aircraft  nose  is  pitched  up)  to  reduce  the 
airspeed  for  a  slow  touchdown.  The  pilot  then  con¬ 
verts  the  stored  rotational  energy  in  the  spinning 
main  rotor  back  to  lift  and  gently  lands  the  helicop¬ 
ter.  The  typical  autorotation  rate  of  descent  at 
touchdown  is  lower  than  a  normal  landing  of  a 
scheduled  airline  landing  on  the  runway.  The  heli¬ 
copter  forward  airspeed  at  touchdown  will  typically 
vary  from  0  to  5  knots,  so  emergency  landi'1  's  can 
and  have  been  achieved  in  very  small  spaces. 

Powcrad  Flight  Unpowcred  Flight 
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Fig.  1.  Helicopter  autorotation. 
Helicopters  Have  Different  Missions  and  Uses 

Using  National  Transportation  Safety  Board 
(NTSB)  accident  data  for  1982  through  1985  for 
USA-registered  helicopters,  the  type  of  mission  un¬ 
derway  at  the  time  of  accident  was  determined  (Ref. 
2).  As  shown  in  Table  1,  single-piston,  single¬ 
turbine,  and  twin-turbine  helicopters  are  used  in 
the  same  missions  but  in  varying  degrees.  Single¬ 
piston  helicopters  have  a  concentration  in  relatively 
high  risk  areas  of  flight  training,  personal  use,  and 
agricultural  work,  where  relative  low  cost  is  a  driv¬ 
ing  factor.  These  types  of  usage  are  major  contribu¬ 
tors  to  the  safety  record  for  single-piston  helicopters. 
If  twin-turbine  helicopters  performed  similar 


missions  and  were  operated  like  the  single-piston 
helicopters,  the  twin-turbine  helicopter  accident 
rate  would  rise  significantly. 

Table  1.  Helicopter  missions  at  accident  NTSB 
data  1982  - 1985  (%  of  accidents) 


Type  of  Single  Single  Twin  All 
Operations  Piston  Turbine  Turbine  Helicopters 


Personal 

26.2 

24.4 

16.0 

24.9 

Business 

9.4 

23.6 

32.0 

14  9 

Instruction 

21.3 

2.0 

8.0 

14.4 

Executive/ 

0 

5.6 

16.0 

2.4 

corporate 

Agricultural 

29.8 

8.8 

4.0 

21.9 

Observation/ 

5.1 

5.2 

0 

5.0 

survey 

Public  use 

1.1 

4.0 

8.0 

2.3 

Ferry/ 

2.3 

4.8 

16.0 

3.6 

Positioning 

Other  work 

4.8 

21.6 

0 

10.6 

Helicopter  Fleet  Is  a  Mixed  Fleet 

The  U.  S.  FAA  Civil  Aircraft  Registry  for  August 
1990  shows  the  following  distribution  of  helicopters 
(Table  2).  The  34  military  surplus  twin-piston  heli¬ 
copters  on  the  Registry  were  not  included.  However, 
the  numbers  of  aircraft  on  the  Registry  can  be  mis¬ 
leading,  because  it  includes  many  aircraft  that  are 
wrecked,  being  salvaged  for  parts,  under  repair, 
stored,  or  used  as  static  (nonflying)  aircraft.  Thus 
flight  hours  are  a  better  indicator  of  actual  aircraft 
usage.  Flight  hours  by  model  series  were  extracted 
from  the  U.  S.  Federal  Aviation  Administration 
(FAA)  General  Aviation  Activities  and  Avionic  Sur¬ 
vey  annual  reports  for  the  same  time  period.  If  the 
FAA  estimated  flight  hours  for  a  model  for  two  or 
more  years  of  the  5-year  period,  those  flight  hours 
were  used.  The  accidents  of  that  model  series  were 
used  if  flight  hours  occurred  in  the  year  of  the  acci¬ 
dent.  If  no  hours  or  one  year  of  flight  hours  were  es¬ 
timated  by  the  FAA  reports,  the  accidents  and 
flights  hours  for  those  affected  models  were  deleted 
from  the  study.  The  author  considers  this  data  to  be 
the  best  available  and  therefore  has  used  it  in  this 
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Table  2.  USA-registered  helicopters  by  engine 
type  (FA A  data) 


Type  of  Engine 

Number  of 
Helicopters 
(11-31-90 
Registry) 

Flight- 
hours  flown 

1984-1988 

Flight- 

hours 

(%) 

Single  piston 

5,371 

2,961,252 

25.9 

Single  turbine 

3,642 

7,035,846 

61.5 

Twin  turbine 

1,108 

1,442,116 

12.6 

Total 

helicopters 

10,121 

11,439,214 

100 

Aircraft  with 
most  flight- 
hours:  206 
single  turbine 
only 

2,092 

5,215,001 

45.6 

paper.  The  usable  models  with  flight  hours  were 
then  arranged  in  groups:  single-piston,  single¬ 
turbine,  and  twin-turbine  helicopters,  and  the  most 
common  helicopter,  the  Model  206.  The  Model  206 
flew  45%  of  all  helicopter  flight  hours  during  the 
1984  through  1988  time  period.  The  Model  206  is 
the  most  prominent  model  and  is  used  as  a  standard 
by  which  other  helicopters  are  typically  compared. 
The  single-turbine  engine  Model  206  will  be  shown 
by  itself  as  well  as  being  in  the  generic  single¬ 
turbine  fleet  throughout  this  paper. 


The  Canadian,  UK,  and  USA  helicopter  fleet  flight- 
hours  shown  in  Table  3  indicate  that  these  helicop¬ 
ter  fleets  are  also  varied.  The  Canadian  accident 
and  flight  hour  data  from  the  Transportation  Safety 
Board  of  Canada  and  Canadian  Aviation  Statistics 
Centre  were  for  the  period  1982  through  1987.  The 
United  Kingdom  accident  data  and  flight  hours 
from  the  Civil  Aviation  Agency  were  for  the  period 
1980  through  1987.  The  mixture  of  the  UK  fleet  fly¬ 
ing  is  significantly  different  from  that  in  Canada 
and  the  USA.  This  helps  to  explain  why  attitudes 
and  helicopter  usages  vary  among  ICAO  States. 
The  most  common  helicopter  flying  in  the  UK  was 
the  S-61  twin  turbine,  which  accounted  for  28.2%  of 
the  UK  flight-hours  whereas  the  Model  206  made  up 
12.3%. 

Disregarding  home-built  and  experimental  helicop¬ 
ters,  it  is  estimated  that  of  approximately  15,200  ro- 
torcraft  in  the  world  (excluding  the  Soviet  bloc 
states)  that  12,51 1,  or  82%,  of  these  rotorcraft  are  in 
the  USA,  the  UK,  and  Canada.  Thus  the  conclu¬ 
sions  from  these  data  should  be  applicable  to  the  re¬ 
maining  helicopters  in  the  non-Soviet-bloc  world. 
The  helicopter  data  are  presented  by  configuration 
groups  of  single  piston  (SP),  single  turbine  (ST),  and 
twin  turbine  (TT). 

MEASURING  SAFETY 

Now  that  we  have  some  indication  of  the  helicopter 
activities,  the  next  step  is  to  measure  safety  or  de¬ 
termine  relative  risk.  There  are  various  methods 
used;  some  are  useful  and  others  are  misleading. 
Using  the  total  number  of  accidents  that  have 


Table  3.  USA,  UK,  and  Canada  civil  helicopters  by  engine  type  (flight-hours  flown) 


Engine  Type 

USA 

(84  -  88) 

UK 

(80-87) 

Canada 

(82  -  87) 

USA /UK/ 

Canada 

Combined 

% 

Single  piston 

2,961,252 

91,737 

190,894 

3,243,883 

21.3 

Single  turbine 

7,035,846 

239,548 

2,078,376 

9,353,770 

61.5 

Twin  turbine 

1,442,116 

932,474 

242,696 

2,617,286 

17.2 

Total  helicopters 

11,439,214 

1,263,759 

2,511,966 

15,214,939 

100 

Most  common 
aircraft:  206  single 
turbine 

5,215,001 

155,648 

1,471,675 

6,842,324 

45.0 
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occurred  for  a  model  is  probably  the  most  mislead¬ 
ing  method  This  primitive  method  does  not  account 
for  fleet  size  and  subsequent  usage/exposure  over 
the  years  and  should  be  avoided.  Accident-per- 
amount-of-exposure  methods  are  more  appropriate. 

Accidents  per  Fleet  Ratio 

One  attempt  to  address  the  effects  of  fleet  size  is  to 
determine  the  ratio  of  accidents  to  the  size  of  fleet  in 
existence  at  the  time  of  comparison.  This  approach 
is  only  slightly  better  than  counting  accidents.  The 
ratio  is  determined  by  counting  the  number  of  acci¬ 
dents  that  have  occurred  on  a  specific  model  in  the 
USA  since  its  introduction.  The  total  accident  histo¬ 
ry  number  is  then  divided  by  the  latest  "estimated” 
number  of  active  helicopters  of  that  model  in  the 
USA.  The  ratio  technique  is  inaccurate  and  mis¬ 
leading  because  it  (1)  disregards  the  changing  fleet 
size  over  the  years,  by  only  using  the  latest  year’s 
"active”  fleet,  (2)  looks  at  models  in  different  periods 
of  their  service  life,  and  (3)  disregards  the  different 
amount  of  flying  done  by  various  models.  Also,  the 
number  of  accidents  will  increase  as  a  model  fleet 
continues  in  use.  In  Figure  2,  the  Model  47,  which  is 
the  oldest  civil  helicopter  model,  suggests  what  may 
happen  to  all  other  models  as  they  mature  in  the  fu¬ 
ture.  The  number  of  accidents  from  1958  through 
1963  were  estimated  from  accident  trends  before 
and  after  that  period.  Since  the  number  of  "active” 
helicopters  are  seldom  known,  the  actual  numbers 
of  civil  aircraft  delivered  with  a  U  S.  Registry  num¬ 
ber  were  used.  Note  that  the  last  Model  47  was  de¬ 
livered  in  1973  in  the  USA.  The  total  number  of  ac¬ 
cidents  grows  each  year  and  far  exceeds  the  number 
of  aircraft  delivered.  Obviously,  the  ratio  of  total  ac¬ 
cidents  to  an  existing  fleet  is  going  to  be  different  de¬ 
pending  on  when  that  ratio  is  calculated.  If  the  ratio 
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Fig.  2.  Model  47  accident/fleet  ratio. 


is  determined  within  two  years  of  model  introduc¬ 
tion,  the  ratio  will  probably  be  low.  Five,  ten,  fifteen 
years  later,  the  ratio  continues  to  increase  regard¬ 
less  of  the  true  model  safety.  Also  shown  in  Figure  2 
is  the  annual  accident  rate  per  100,000  flight  hours. 
Note  that  the  accident  ratio  continues  to  climb  to 
about  1 60%  as  of  1 985  even  though  the  accident  rate 
is  basically  decreasing  over  the  last  three  years. 
This  disparity  will  be  present  for  all  other  models 
and  is  dependent  on  when  in  the  model’s  life  cycle 
the  ratio  is  computed.  » 


When  comparing  vastly  different  types  of  aircraft,  it 
was  apparent  that  some  types  spent  the  majority  of 
their  flight  time  in  the  more  hazardous  flight  phases 
of  takeoffs  and  landings.  Thus  the  accident  rate  per 
departure  (or  mission)  was  used.  This  approach  an¬ 
swers  the  question  "Is  the  likelihood  of  this  mission 
failing  greater  or  lesser  for  Means  A  vs.  Means  B?” 
but  is  not  concerned  with  how  long  Means  A  or 
Means  B  takes  to  accomplish  the  mission.  For  ex¬ 
ample,  if  the  mission  is  to  get  a  person  from  Point  X 
to  Point  Y,  the  following  means  of  travel  can  qualify 
for  the  task. 

-Rocket 

-Jet  airplane 

-Helicopter 

-Train 

-Automobile 

-Boat 

-Balloon 

-Walking. 

The  number  of  accidents  that  occurred  from  the  time 
of  departing  Point  X  and  arriving  at  Point  Y  would 
then  be  determined  for  each  means  of  travel.  When 
divided  by  the  number  of  missions  attempted,  this 
becomes  the  accident  rate  per  departure. 

Helicopters  can  perform  some  missions  which  no 
other  transportation  means  can  achieve.  These 
unique  missions,  involving  hovering  or  very  slow 
flight,  cause  very  short  flight  times,  and  result  in  a 
large  number  of  takeoffs  and  landings.  Since  large 
airplanes  spend  the  vast  majority  of  their  flight  time 
in  cruise  rather  than  in  takeoff/landings,  compari¬ 
sons  with  helicopters  are  somewhat  biased.  A  study 
in  1981  included  a  look  at  Part  135  unscheduled  air- 
taxi  helicopter  safety  related  to  airplane  air  carriers 
(Ref.  3).  The  helicopter  operators  surveyed  flew  603 
single-  and  twin-turbine  helicopters  during  the 


Accidents  per  Departure 
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subject  period  (1977  through  1979).  The  percentage 
of  singles  vs.  twins  is  no  longer  available,  however, 
the  percentage  of  single  turbines  vs.  twin  turbines  is 
available  for  1983,  which  is  the  nearest  period.  The 
1983  U.S.  Registry  indicates  a  mix  of  83%  single 
turbines  and  17%  twin  turbines.  The  mix  in  the 
helicopter  survey  group  should  have  been  similar 
The  accident  rate  per  flight  hour  for  the  combined 
turbine  helicopter  fleet  compared  to  the  air  carriers 
is  shown  in  Figure  3A.  This  shows  that  the 


Helicopter  U.S.  Commuter 

operators  certificated  air  carriers 

air  carriers 


helicopter  accident  rate  per  flight  hour  was  slightly 
better  than  that  of  commuter  (regional)  air  carriers. 
To  account  for  time  spent  in  the  more  hazardous 
phases  of  flight  ( i.e.,  takeoff  and  approach/landing), 
the  accident  frequency  is  based  on  number  of  depar¬ 
tures  (i.e.,  takeoffs).  The  heticopter  accident  rate 
per  100,000  departures  was  73%  lower  than  com¬ 
muter  air  carriers,  as  shown  in  Figure  3B.  The  heli¬ 
copter  rate  was  much  closer  to  the  rate  for  the  large 
certificated  air  carriers.  Figure  3C,  fatal  accident 


operators  certificated  air  carriers 
air  carriers 


Fig3A.  Accident  rate  per  flight  hour  Fig3B.  Accident  rate  per  departure 
(all  causes).  (all  causes). 


operators  certificated  air  carriers  operators  certificated  air  carriers 

air  carriers  air  carriers 

Fig3C.  Fatal  accident  rate  per  Fig3D.  Numberof  fatalities  per 

departure  (all  causes).  departure  (all  causes). 


*  Part  135  helicopter  operators  survey. 

*•  National  Transportation  Safety  Board,  Special  Study, 

Commuter  Airline  Safety  (Reference  4). 

INlll 

Fig.  3.  Accident  and  fatality  rate  comparison  (1977  -  1979). 
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rate  per  departure,  shows  that  the  helicopter  rate 
was  69%  lower  than  the  commuter  air  carriers. 
Figure  3D  shows  comparable  data  for  fatalities  per 
departure.  In  this  case,  the  helicopter  rate  is  71% 
lower  than  commuter  air  carriers  and  79%  lower 
than  certificated  air  carriers.  The  helicopter 
industry  in  general  is  safer  than  perceived  by  those 
outside  of  this  industry,  considering  the  amount  of 
time  spent  in  hazardous  phases  of  flight.  This  also 
indicates  the  variability  of  potential  safety 
perceptions,  depending  on  the  method  of 
measurement. 

The  offshore  oil  industry  in  the  Gulf  of  Mexico  gives 
a  good  indication  of  the  safe  operation  of  turbine  he¬ 
licopters  (Table  41.  In  1990,  there  were  1,855,345 
takeoffs  and  landings.  About  1,500,000  of  these 
takeoffs  and  landings  were  at  offshore  platforms. 
There  were  3,958,525  passengers  moved  by  helicop¬ 
ter.  Of  the  619  helicopters  in  the  Gulf  of  Mexico,  138 
(22%)  are  IFR  equipped.  Single-turbine  helicopters 
account  for  349  (56%)  of  the  total  helicopter  fleet. 
This  significant  usage  of  single-turbine  helicopters 
indicates  that  single  turbines  are  being  operated 
safely  from  elevated  platforms  and  over  water 

This  method  based  on  departure  exposure  is  accu¬ 
rate  for  determining  the  risk  to  mission  accomplish¬ 
ment,  but  is  not  accurate  for  determining  safety. 
Safety  is  related  to  "freedom  from  harm,  injury,  or 
loss  and  should  be  counted  in  terms  of  time  of  indi¬ 
vidual  occupant  exposure. 

Accidents  per  Patient  Transport 


transported  during  the  same  time  period.  This 
unique  approach  uses  the  EMS  primary  function  of 
"moving  patients”  as  the  basis  for  comparison  with 
the  safety  of  other  modes  of  "moving  patients.”  This 
approach  is  appropriate  for  only  that  medical  trans¬ 
port  mission  comparison  of  mission  completion,  not 
safety  of  the  crew  and  patient.  Figure  4  (from  Ref.  5) 
shows  the  annual  EMS  helicopter  accident  rates  per 
100,000  patients  transported.  It  cannot  be  used  to 
compare  with  "non-patient-carrying”  aircraft.  Since 
many  of  the  EMS  helicopter  accidents  occurred 
without  a  patient  onboard  (e  g.,  en  route  to  pickup, 
returning  after  transport,  or  repositioning),  this  is  a 
mission-oriented  measurement  (similar  to  "per  de¬ 
parture”),  rather  than  "per  human”  exposure. 
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Fig.  4.  EMS  accidents  per  patient 
transport  operation. 

Accidents  per  Passenger  Mile 


This  recent  safety  measurement  variation  is  used  by 
the  emergency  medical  services  (EMS)  community. 
This  accident  rate  is  the  number  of  EMS  aircraft  ac¬ 
cidents  that  occur  divided  by  the  number  of  patients 


Accidents  per  passenger  mile  is  another  "per  mis¬ 
sion”  measurement,  with  an  adjustment  for  the  dis¬ 
tance  traveled.  Fixed-wing  scheduled  air  carriers 
and  fixed-  and  rotary-wing  air  taxi  operators  have 


Table  4.  Gulf  of  Mexico  helicopter  safety  data 


Year 

Fleet  Size 

No.  of 
Accidents 

Flight- 

hours 

Departures 

Accidents  per 
100,000 
Departures 

Accidents 

per 

100,000 

Flight- 

hours 

1987 

708 

17 

691,655 

2,101,850 

0.80 

2.46 

1988 

599 

10 

455,330 

1,384,000 

0.65 

2.20 

1989 

608 

9 

515,770 

1,885,571 

0.48 

1.74 

1990 

619 

9 

533,761 

1,855,345 

0.49 

1.69 
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passenger-carried  information  from  revenue  flights; 
but  General  Aviation  and  helicopters,  in  general,  do 
not.  Thus  comparisons  are  seldom  made  in  this 
area.  Limitations  of  "per  mission”  measurement  are 
easily  noted  by  comparing  the  safety  of  an  80-knot 
aircraft  with  a  400-knot  aircraft,  both  having  the 
same  number  of  passengers  and  accidents  per  pas¬ 
senger  mile.  Some  people  try  to  interpret  this  as  the 
same  level  of  occupant  safety.  However,  the  slower 
machine  is  in  the  air  five  times  as  long  as  the  faster 
aircraft  for  the  same  distance.  Therefore,  the  slower 
aircraft  must  have  only  one  fifth  of  the  accident  rate 
per  flight  hour  of  the  faster  aircraft.  This  dichotomy 
is  due  to  the  primary  concern  being  "per  mission” 
and  not  related  to  "per  human”or  occupant  safety. 
Accident  per  passenger  mile  is  only  meaningful  if 
your  primary  concern  is  mission  completion  of  mov¬ 
ing  a  passenger  a  given  distance,  not  the  safety  of 
the  occupants. 

Accidents  per  Flight-hours 

The  most  common  method  presently  used  is  "acci¬ 
dent  rate  per  100,000  flight  hours.”  This  accident 
rate  per  hour  is  the  number  of  accidents  of  a  model 
for  a  specific  period  of  time  divided  by  the  hours 
flown  by  those  aircraft  over  the  same  time  period. 
This  accident  rate  per  100,000  flight  hours  is  an 
good  method  to  determine  the  aircraft  damage  cost 
expected  in  a  model  fleet  or  the  likelihood  of  aircraft 
damage.  Table  5  shows  the  accident  rates  per 
100,000  flight  hours  for  USA  General  Aviation 
fixed-wing  and  rotary-wing  aircraft  in  descending 
order. 

Helicopter  accident  rates  for  the  1980s  from  the 
USA,  the  UK,  and  Canada  for  the  time  periods  of 
Table  3  are  shown  in  Table  6. 

AIRWORTHINESS  VS.  OPERATIONAL 
ISSUES 

The  causes  of  accidents  resulting  in  serious  (major/ 
fatal)  occupant  injury  were  determined  (Ref.  6)  us¬ 
ing  NTSB  data  from  1982  through  1986  for  single¬ 
turbine  and  twin-turbine  civil  helicopters,  as  shown 
in  Figure  5.  Engine  material  failure  (MF)  initiated 
the  crashes  that  caused  14.8%  of  the  serious  ir\juries 
to  occupants  of  single-turbine  helicopters,  as  com¬ 
pared  to  only  3.4%  for  the  serious  injuries  to  occu¬ 
pants  of  twin-turbine  helicopter  accidents.  If  you 
stop  here  with  only  this  one  piece  of  information,  the 
obvious  conclusion  is  that  two  is  better  than  one. 
However,  consider  only  material  failures  other- 


Tabie  5.  USA-registered  general  aviation  acci- 

nnnt ppfpc 

(NTSB/FAA  data  1984  - 1988) 


Type  of  Aircraft 

Accidents 
per  100,000 
night-hours 

Single-piston  helicopter 

17.83 

Single-piston  airplane 

8.55 

Single-turbine  helicopter  (all) 

5.49 

Twin-piston  airplane 

5.12 

Twin-turbine  helicopter' 

4.37 

Bell  Model  206  single-turbine 
helicopter 

4.28 

Table  6.  1980s  USA,  UK,  and  Canadian 

accident  rates  (all  causes) 
(accidents  per  100,000  flight-hours) 


Type  of 
helicopter 

Canada 

(82-87) 

United 

Kingdom 

(80-87) 

United 

States 

of 

America 

(84-88) 

Single  piston 

33.53 

73.79 

17.83 

Single  turbine 
(all) 

9.86 

17.12 

5.49 

Twin  turbine 

4.67 

4.83 

4.3? 

Model  206 
single  turbine 

8.70 

14.07 

4.28 

SJngla-Turbm*  Twin-TiwbiM 

Hatkoptars  Halkoptart 


Parcantag*  of  injuries  by  caws*  of  crash**  (NTS!  Kt-M) 

i-NIIS 

Fig.  5.  Seriously  injured  occupants  by 
accident  cause. 

than-engine  or  non-engine  MF.  Only  11.0%  of  the 
seriously  injured  occupants  were  in  single-turbine 
helicopter  crashes  initiated  by  non-engine  material 
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failures  as  compared  with  31.0%  in  twin-turbine 
helicopter  crashes.  This  is  an  indicator  of  the  detri¬ 
mental  effects  of  complexity  and  more  parts  If  you 
were  to  consider  only  this  last  piece  of  information, 
the  obvious  approach  should  be  to  ban  all  twin-tur¬ 
bine  helicopters  and  only  use  single-turbine  heli¬ 
copters.  Actually,  the  total  material  failures,  en¬ 
gine  and  non-engine,  should  be  considered  together, 
which  yields  percentages  of  seriously  injured  occu¬ 
pants  due  to  all  types  of  MF-caused  accidents  of 
25.8%  for  occupants  in  single  turbines  and  34.4%  for 
occupants  in  twins.  This  is  consistent  with  the 
greater  number  of  parts  and  increased  complexity 
present  in  twins.  Since  deaths  or  injuries  do  not 
only  occur  as  a  result  of  engine-related  factors,  it  is 
essential  that  all  other  factors  be  considered  as  well, 
both  material  failure  and  nonmaterial  failure  (i.e., 
human  error).  The  accident  rates  for  the  combined 
U.S.  helicopter  fleet  (all  helicopters)  and  the 
individual  types  are  shown  in  Table  7. 

Engine  material  failures  are  just  one  of  the  material 
failures  (also  called  airworthiness  failures)  that 
cause  accidents;  the  remaining  non-engine  material 
failures  that  caused  accidents  are  also  shown  in  Ta¬ 
ble  7.  The  single-piston  accident  rate  per  100,000 
flight-hours  for  non-engine  material  failure 
accidents  is  the  highest  rate,  followed  by  twin 
turbines,  all  single  turbines,  and  (with  the  lowest 


rate)  the  Model  206  single  turbine.  Table  7  shows 
the  combined  engine  and  non-engine  material 
failures  (eg.,  all  airworthiness  failures),  and 
indicates  that  the  accident  rate  for  all  airworthiness 
failures  in  twin  turbines  is  significantly  lower  than 
for  single  pistons,  and  slightly  lower  than  for  all 
single  turbines,  but  still  51.4%  higher  than  the 
single-turbine  206  rate.  From  an  overall 
airworthiness  standpoint,  there  is  no  justification  to 
require  twin-turbine  engines  on  ALL  helicopters  for 
ALL  mission  applications. 

Statistical  Significance 

Individual  yearly  airworthiness  failure  accident 
rates  will  vary  from  year  to  year  due  to  the  random 
natures  of  rare  events  like  accidents,  as  shown  for 
turbine  helicopters  in  Table  8.  The  statistical  sig¬ 
nificance  of  the  single-  and  twin-turbine  helicopter 
accident  rates  was  used  to  determine  if  the  rates  for 
all  their  accidents  due  to  airworthiness  failures 
were  significantly  different  or  not.  The  statistical 
method  used  for  this  determination,  "Student  T,” 
utilized  a  level  of  significance  of  0.05.  This  tech¬ 
nique  can  determine  the  likelihood  that  the  two  sets 
of  data  (i.e.,  accident  rates  of  singles  vs.  twins)  will 
be  from  the  same  group  (i.e.,  not  of  significant  differ¬ 
ence),  with  the  observed  rate  varying  only  due  to 
chance.  A  level  of  significance  of  0.05  indicates  that 


Table  7.  USA-registered  helicopter  accident  rates  (Sources:  NTSB/FAA  for  1984  through 
1988)  (Accidents  per  100,000  flight-hours) 


Type  of  Helicopter 

Engine-only 

Airworthiness 

Non-engine 

Airworthiness 

All 

Airworthiness 

All  Causes 

All  helicopters 

1.22 

1.08 

2.30 

8.54 

Single  piston 

1.99 

2.09 

4.09 

17.83 

Twin  turbine 

0.35 

1.25 

1.59 

4.37 

Single  turbine  (all) 

1.08 

0.61 

1.69 

5.49 

206  single  turbine 

0.88 

0.17 

1.05 

4.28 

Table  8.  USA  turbine  helicopter  fleet  airworthiness-failure  annual  accident  rates  (NTSB/FAA  data  for 
1984-88)  (Accidents  per  100,000  flight-hours) 


Type  of  Helicopter 

1984 

1985 

1986 

1987 

1988 

Average 

(84-88) 

Single-turbine  helicopter  (all) 

1.65 

1.95 

2.04 

1.59 

1.30 

1.69 

Twin-turbine  helicopter 

1.76 

0.95 

2.14 

1.92 

0.98 

1.59 

Bell  Model206  single-turbine  helicopter 

0.95 

1.46 

1.21 

0.79 

0.92 

1.05 
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the  statement  being  made  will  be  wrong  no  more 
than  5  times  out  of  100.  In  other  words,  the  state¬ 
ment  being  made  will  be  correct  95  times  out  of  100. 
The  airworthiness-failure  accident  rates  of  singles 
and  twins  are  not  significantly  different  95  times  out 
of  100. 

Comparing  the  all-airworthiness-failure  accident 
rates  of  the  three  ICAO  States  (the  USA.,  the  UK, 
and  Canada)  are  Table  9  and  Figure  6,  which  show 
the  variability  that  is  a  function  of  the  mix  of  air¬ 
craft  models  within  a  type  and  how  they  are  used  in 
the  different  ICAO  States.  The  rates  of  twin  turbi¬ 
nes  and  Model  206s  appear  to  be  quite  consistent.  It 
is  interesting  to  note  that  the  single-turbine  Model 
206  has  the  lowest  airworthiness  accident  rate  in 
two  of  the  three  ICAO  States  and  second  lowest  in 
the  remaining  State.  These  data  do  not  justify  the 
ICAO  Annex  6,  Amendment  1  prohibition  of  single¬ 
engine  helicopters. 


Table  9.  1980s  USA,  UK,  and  Canadian ' 

airworthiness-failure  accident  rates 
(Accidents  per  100,000  flight-hours) 


Type 

Canada 

(82-87) 

United 
Kingdom 
(80  -  87) 

United 

States 

of 

America 

(84-88) 

Single  piston 

8.91 

18.45 

4.09 

Single  turbine 
(all) 

2.12 

4.17 

1.69 

Twin  turbine 

1.27 

1.93 

1.59 

Bell  206  single 
turbine 

1.43 

1.17 

1.05 

Time-Exposed  Comparisons 


Accidents  are  quite  rare  events.  A  trip  around  the 
world  at  the  equator  would  take  about  192.3  hours 
for  a  turbine  helicopter  flown  at  an  average  cruise 
speed  of  130  miles  per  hour  (113  kn)  Thus,  a 
comparison  of  the  mean-time-between-accidents 
(MTBA)  can  be  expressed  in  an  exposure-time 
equivalent  of  around-the-world  trips.  The  MTBA  is 
the  number  of  hours  flown  by  an  aircraft  type 
divided  by  the  number  of  accidents  (e.g.,  the  inverse 
of  the  accident  rate). 

The  around-the-world  trip  equivalent  for  the  Model 
206  and  twin-turbine  helicopters  shows  the 


Single  piston  Single  turbine  Twin  turbine  Sell  206 


1-N114 

Fig.  6.  Airworthiness  failure  accident 

rates  for  UK,  USA,  and  Canada  in 
the  1980’s. 

remoteness  of  accidents  from  all  airworthiness  fail¬ 
ures  (both  engine  and  non-engine).  Using  the  data 
from  Table  7,  the  MTBA  for  the  Model  206  and  a 
twin-turbine  helicopter  can  be  derived  as  95,283 
hours  and  62,893  hours,  respectively.  Thus,  on  the 
average,  the  expectancy  of  an  accident  due  to  air¬ 
worthiness  failure  was  equivalent  to  495  trips  and 
327  trips  for  the  Model  206  and  for  twin-turbine  air¬ 
craft,  respectively.  Likewise,  the  number  of  years 
for  one  aircraft  to.  fly  those  around-the-world  trips, 
without  landing  (i.e.,  continuous  24-hr/day  flying) 
would  be  about  10.9  years  and  7.2  years  for  the 
Model  206  and  a  twin-turbine  helicopter,  respective¬ 
ly.  The  chance  of  an  accident,  for  both  types,  is  ex¬ 
tremely  remote.  There  is  no  safety  justification  for 
prohibiting  single-engine  or  twin-engine  helicopters 
from  flying  over  congested  areas  or  hostile-earth 
surfaces. 

: :  < 

Accident  Site  Surface 

A  failure  of  the  engine  does  not  automatically  mean 
that  an  accident  will  occur.  The  type  of  terrain  can 
influence  whether  the  results  of  autorotation  wiU  be 
merely  a  forced  landing  or  an  accident  with  damage. 
If  the  engine  fails  over  terrain  hospitable  to  an 
emergency  landing,  such  as  prepared  hard  surfaces, 
unprepared  ground,  soil,  fields,  open  terrain,  or  heli¬ 
pads,  then  autorotation  is  possible  without  further 
damage,  and  no  accident  occurs.  If  the  helicopter  is 
equipped  with  an  aircraft  flotation  system,  an  emer¬ 
gency  landing  can  be  made  on  water.  Such  a  water 
landing  is  not  considered  an  accident  unless  signifi¬ 
cant  aircraft  damage  or  an  injury  occurs;  with  signi¬ 
ficant  damage  or  injury,  it  is  considered  an  accident. 
If  the  terrain  is  inhospitable  (e.g.,  covered  with 
trees,  swamps,  or  walls),  the  emergency  landing 
may  result  in  an  accident. 
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The  known  impact  surfaces  for  all  U.S.  civil  helicop¬ 
ter  accidents  from  1984  through  1988  are  shown  in 
Table  10.  Accidents  from  all  causes  are  included. 
Unknown  site  surfaces,  inflight  breakups,  midair 
collisions,  and  unknown  types  of  ground  surfaces, 
accounting  for  about  28%  of  the  accidents,  were  de¬ 
leted  as  nonusable.  The  surface  category 
"Trees/swamp/wall”  includes  terrain  where  a  suc¬ 
cessful  emergency  landing  without  damage  is  not 
likely  after  a  power  loss  or  any  other  immediately 
required  emergency  landing.  This  was  the  surface 
for  10.5%,  10.7%,  and  8.8%  of  the  impacts  for  twin 
turbines,  single  turbines,  and  the  Model  206,  respec¬ 
tively.  Thus,  the  accident  history  for  impacts  into 
trees  (inhospitable  sites)  has  not  been  different  for 
twin-turbine  than  for  single-turbine  helicopters. 
Care  should  be  taken  in  reading  Table  10,  as  the  ex¬ 
posure  over  these  surfaces  has  not  been  the  same  for 
each  type  of  aircraft.  The  value  of  many  helicopter 
jobs  over  inhospitable  terrain  cannot  justify  the  use 
of  an  expensive  helicopter,  and  therefore  a  small, 
less  expensive  helicopter  is  often  used.  The  table  is 
indicative  of  how  aircraft  are  being  used,  rather 
than  pointing  out  relative  danger  of  impact  sites. 
For  example,  the  twin-turbine  helicopter  had  33.3% 
of  its  known-site  accidents  in  the  "Prepared  sur¬ 
face/pad”  category,  which  is  the  least  dangerous  im¬ 
pact  site.  Overall,  the  table  shows  that  basically  all 
types  of  impact  sites  have  occurred  with  all  types  of 
helicopters. 

Fatal  Accidents  per  Flight-hours 

Safety  is  typically  defined  as  a  condition  of  freedom 
from  harm,  injury,  or  loss.  Thus,  measurement  of 
those  accidents  involving  fatal  injuries  is  relevant  to 
the  relationship  of  safety  to  human  suffering.  A 
fatal  accident  is  an  accident  in  which  at  least  one 
person  is  fatally  injured.  A  fatal  accident  rate  is  the 
number  of  fatal  accidents  per  100,000  flight  hours. 
Table  1 1  shows  the  fatal  accident  rate  for  the  three 
types  of  helicopters  in  the  USA.  These  aircraft  types 
have  about  the  same  fatal  accident  rate.  This 
method  is  still  inaccurate,  as  it  does  not  account  for 
the  number  of  people  onboard  that  had  the  chance  of 
being  fatally  injured.  For  example,  for  a  twenty- 
place  helicopter  with  ten  people  onboard,  there  is 
five  times  the  chance  of  someone  being  killed  as  for  a 
five-place  helicopter  with  two  people  onboard.  This 
is  due  to  the  difference  in  ten  people  impacting  the 
ground  in  one  airframe  vs.  two  people  in  the  other 
airframe.  Obviously,  the  number  of  helicopter  seats 
is  not  important;  but  the  number  of  people  onboard 
is  important.  Thus,  fatal  accident  rates  are 


Table  10.  Known  accident  impact  sites 


Type  of  Impact 
Surface 

Model  206 
Single 
Turbine 
(%) 

All 

Single 

Turbines 

(%) 

Twin 

Turbines 

(%) 

Clearing/ 

brush/burm 

6.4 

5.4 

0 

Rough  ground/ 
rocks 

11.1 

10.7 

0 

Ground/soil/ 

unprepared 

23.4 

31.9 

12.3 

Prepared 

surface/pad 

19.3 

17.1 

33.3 

Trees/swamp/ 

wall 

8.8 

10.7 

10.5 

Buildings 

1.2 

1.0 

3.5 

Auto/boat/ 

railroad 

0.6 

0.6 

3.5 

Snow/ice 

2.9 

3.7 

1.8 

Water 

24.0 

17.4 

29.8 

Rig 

2.3 

1.3 

5.3 

Table  11.  USA  helicopter  fatal  accident  rates 
(NTSB/FAA  1984 -1988) 


Fatal  accidents  per 
Type  of  Helicopter  100,000  flight-hours 


Single  piston 

1.89 

Twin  turbine 

1.10 

Single  turbine 

1.08 

misleading  when  they  are  related  to  aircraft  air¬ 
frame  accidents,  not  to  the  occupants.  Fatal  acci¬ 
dent  rates  should  not  be  used  to  measure  safety. 


Risk  around  Heliports 

Some  neighbors  around  heliports  have  voiced  con¬ 
cern  about  safety  of  helicopters  approaching  or  leav¬ 
ing  a  heliport.  These  concerns  are  unfounded.  The 
actual  risk  to  the  neighborhood  from  helicopters  was 
analyzed  (Ref.  7)  to  determine  the  likelihood  of  a 
helicopter  accident  in  a  0.8-km  ( 1/2-mile)  radius  of 
the  heliport/airport  using  NTSB/FAA  data  for  the 
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period  of  1975  through  1978.  This  analysis  was 
based  on  the  Model  206  accident  rate  of  4.33/100,000 
flight-hours.  A  3-minute  time  period  spent  over  the 
0.8-km  (1/2-mile)  radius  for  an  approach  or  landing 
was  used  to  be  conservative.  One  can  then  calculate 
the  likelihood  of  an  accident  within  the  0.8-km  (1/2- 
mile)  radius  which  becomes  a  function  of  how  many 
takeoffs  and  landings  a^e  made.  The  term  "cycle”  is 
used  for  the  combination  of  a  takeoff  and  landing 
(e.g.,  6  minutes  over  the  0.8-km  (1/2-milel  zone). 
Using  the  average  number  of  cycles  per  day  for  a 
year,  the  average  number  of  years  between  acci¬ 
dents  can  be  determined  using  Figure  7.  For  exam¬ 
ple,  for  a  busy  heliport  conducting  5  cycles  per  day 
(182.6  hours  per  year  over  the  0.8-km  [1/2-mile] 
zone),  the  expected  average  of  years  between  acci¬ 
dents  should  be  128  years.  One  accident  in  128 
years  is  an  extremely  remote  possibility. 


Fig.  7.  Helicopter  accidents  within  0.8  km 
(1/2  mile). 


Likewise,  the  likelihood  of  a  helicopter  striking  a 
residence  or  building  within  a  0.8-km  (1/2-mile)  ra¬ 
dius  of  a  heliport  can  be  estimated  using  Figure  8. 
The  accident  frequency  used  was  for  all  helicopter 
accidents  (i.e.,  single  piston,  single  turbine,  and 
twin  turbine)  involved  in  striking  a  residence  or 
building.  For  the  5-cycle-per-day  case,  a  helicopter 
striking  a  building/residence  is  estimated,  on  aver¬ 
age,  once  every  4,000  years.  This  is  extremely  re¬ 
mote.  Figure  9  shows  the  likelihood  of  an  on-the- 
ground  person  (i.e.,  not  a  crewman  or  passenger)  be¬ 
ing  injured  within  this  0.8-km  (1/2-mile)  radius.  For 
the  5-cycle-per-day  case,  this  shows  that  the  average 
number  of  years  between  injuries  to  be  about  5,000 
years.  This  is  likewise  extremely  remote. 

The  heliport  operating  at  5  cycles  per  day  over  a  one- 
year  period  is  an  extremely  busy  heliport.  For  a 


6  .  ' 

Fig.  8.  Helicopter  strikes  residence  or 

building  (within  0.8-km  radius). 


Fig.  9.  On-the-ground  personal  injury 
(within  0.8-km  radius). 


private  heliport  or  limited  use  that  averages  less 
than  1  cycle  per  day  over  each  year  period,  the  risk 
is  significantly  lower.  Using  1  cycle  per  day  av¬ 
erage,  the  likelihood  of  an  accident  in  the  0.8-km 
(1/2-mile)  area,  the  likelihood  of  striking  a 
residence/building,  and  the  likelihood  of  an  on-the- 
ground  person  being  injured  are  once  in  635  years, 
22,400  years,  and  25,000  years,  respectively.  These 
average  year  values  in  themselves  are  not  impor¬ 
tant  but  their  magnitudes  indicate  the  extremely  re¬ 
mote  threat  due  to  helicopters  operating  over  a  con¬ 
gested  area. 

If  only  airworthiness-failures-caused  accidents  are 
considered  using  the  Model  206  and  twin-turbine 
helicopter  rates  of  Table  7,  a  comparison  of  the  like¬ 
lihood  of  an  airworthiness-caused  accident  over  the 
neighborhood  can  be  made.  For  a  constant  usage  of 
5  cycles  per  day,  the  expected  accident  frequency 
within  the  0.8-km  (1/2-mile)  radius  of  the  heliport  is 
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an  accident  once  in  34.4  and  52.2  years  for  a  twin- 
turbine  helicopter  and  Model  206  single-turbine  he¬ 
licopter,  respectively.  Thus  one  should  expect  the 
Model  206  accident  significantly  less  often  than  the 
twin-turbine  helicopter  accident.  The  likelihood  for 
both  helicopter  types  is  extremely  remote.  There  is 
no  more  justification  to  prohibit  twin-turbine  heli¬ 
copters  than  there  is  to  prohibit  a  Model  206  from  fly¬ 
ing  over  congested  (e.g.,  populated)  areas. 

Causes  of  Accidents  Resulting  in  Fatalities 

A  study  of  Bell  civil  and  military  turbine-powered 
helicopter  accidents  around  the  world  was  conduct¬ 
ed  to  determine  the  accident  causes  that  resulted  in 
fatalities.  The  period  of  time  was  January  1970 
through  March  1987.  The  size  of  the  Bell  turbine 
fleet  delivered  at  the  time  was  approximately  19,700 
single-turbine  aircraft  and  1,800  twin-turbine  air¬ 
craft.  An  engine  failure  was  the  initiating  cause 
that  resulted  in  6%  of  all  fatalities  in  single-turbine 
helicopter  accidents  and  3%  of  all  fatalities  in  twin- 
turbine  helicopter  accidents  as  shown  in  Figure  10. 
However,  the  percentage  of  fatalities  due  to  remain¬ 
ing  airworthiness  failures  (non-engine  material  fail¬ 
ures)  was  12%  and  22%  for  svr.gle-turbine  and 
twin-turbine  helicopters,  respectively.  Thus  the  to¬ 
tal  percentage  of  fatalities  for  all  airworthiness  fail¬ 
ures  was  18%  for  single-turbine  helicopters  and  25% 
for  twin-turbine  helicopters.  It  is  apparent  that 
more  complex  twin-turbine  helicopters  will  have  a 
higher  total  number  of  material  failures  (engine  and 
non-engine)  with  a  corresponding  higher  total  num¬ 
ber  of  fatal  injuries  than  a  simpler  single-turbine 
helicopter. 

OCCUPANT  RISK 

Relative  Risk  of  Serious  Injury 

Accident  rates  compare  the  frequency  of  aircraft  be¬ 
ing  damaged  to  such  an  extent  that  it  must  be  re¬ 
ported  as  an  accident.  In  the  majority  of  accidents, 
there  is  no  serious  injury,  so  the  accident  reporting 
is  basically  an  aircraft  damage  mishap  frequency. 
This  information  is  useful  in  forecasting  the  number 
of  aircraft  expected  to  be  damaged,  repaired,  re¬ 
placed,  or  other  activities  based  on  aircraft  damage. 
It  does  not  address  the  safety  of  the  occupant.  A  per¬ 
son’s  safety  is  a  personal  issue,  applied  on  an  indi¬ 
vidual  basis,  not  an  aircraft  basis.  Risk  must  be 
limited  to  an  individual  occu-~-  1  to  be  meaningful. 
Occupant  safety  must  be  det,ei  mined  for  each  indi¬ 
vidual  occupant  based  on  his  individual  exposure. 


Accident  Initiator 


Single-turbine  failure  Twin-turbine 

Helicopters  Helicopters 


BHTI  turbine  Mkopttn  (970- Mar  t7 
1-NI18 

Fig.  10.  Percentage  of  fatalities  by  accident 
initiator. 

This  is  done  with  Relative  Risk  of  Serious  Injury 
(RSI).  RSI  is  the  probability  of  an  accident  occur¬ 
ring  times  the  probability  of  serious  (e  g.,  major  or 
fatal)  injury.  The  RSI  is  calculated  by 

N  umber  of  N  umber  of  people  with 

accidents  major  or  fatal  injury 

RSI  =  - x - 

Flight-hours  Total  number  of  people 

flown  on  board  in  accidents 

The  RSI  or  an  individual  occupant  risk  of  serious  in¬ 
jury  for  every  100,000  occupant-hours  of  exposure  is 
shown  in  Figure  11  for  all  airworthiness-failure 
causes.  This  is  the  true  measure  of  occupant  safety 
related  to  the  aircraft  design. 


Single-piston  Single-turbine  Twin-turbine  Model  206 
Helicopter  helicopter  helicopter  helicopter 
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Fig.  11.  RSI  from  airworthiness  failures. 

Thus  an  occupant’s  risk  of  a  serious  injury  due  to  ac¬ 
cidents  caused  by  all  airworthiness  failures  is  the 
same  in  the  generic  single-turbine  and  the  twin- 
turbine  helicopters.  An  occupant’s  risk  in  a  Model 
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206  single-turbine  helicopter  is  nearly  half  that  of 
being  in  a  twin-turbine  helicopter.  Based  on  the  risk 
to  helicopter  occupants,  there  is  no  justification  to 
prohibit  the  use  of  single -turbine  helicopters.  The 
reasons  that  risks  are  generally  higher  in  twins 
than  singles  are 

1.  More  parts  and  increased  complexity  yield 
more  non-engine  material  failures,  causing  acci¬ 
dents. 

2.  There  are  more  freestanding  passenger 
seats  and  resulting  seat  failures  in  twins. 

3.  There  are  more  passenger  seats  without 
shoulder  harnesses. 

4.  More  fuel  cells  leads  to  increased  likelihood 
of  post-crash  fires. 

The  introduction  of  passenger  shoulder  harnesses, 
energy-attenuating  seats  for  all  occupants,  and  the 
Crash  Resistant  Fuel  Systems  may  lower  the  RSI. 
FAA  Amendments*  will  require  shoulder  harness 
and  dynamically  tested  energy-attenuating  seats  for 
all  occupants  in  future  helicopter  designs.  A  shoul¬ 
der  harness  is  required  for  all  seat  locations  in  all 
helicopters  manufactured  in  the  USA  or  for  use  in 
the  USA  after  September  16, 1992.**  FAA  Notice  of 
Proposed  Rule  Making  (NPRM)  90-24  in  progress  is 
addressing  a  requirement  to  include  a  Crash  Resis¬ 
tant  Fuel  System  in  large  and  small  helicopters  to 
minimize  thermal  injuries  due  to  post  crash  fires. 
Thus  occupants  of  future  helicopter  designs  may 
have  even  lower  risk  of  serious  injury,  regardless  of 
what  causes  the  accidents. 

A  study  (Ref.  8)  of  U.S.  Army  helicopter  accidents 
and  injuries  found  similar  results  to  the  risk  in  civil 
helicopters.  Table  12  shows  the  RSI  for  the  four 
Army  helicopters  in  the  study.  The  UH-60  is  the 
twin-turbine  helicopter  and  the  *  .mainder  are 
single-turbine  powered.  The  risk  of  injury  was  low¬ 
er  in  the  single-turbine  helicopters  than  in  the  twin 
turbine.  There  are  several  reasons  for  this;  two  of 
these  are  the  greater  complexity  of  the  UH-60  and 
its  higher  impact  speeds.  Again,  one  must  be  care¬ 
ful  to  evaluate  all  aspects  of  an  aviation  system 
since  improvements  in  one  area  can  have  detrimen¬ 
tal  effects  in  another  area.  One  of  safety’s  goals  is  to 
strive  for  the  best  mix  to  get  the  lowest  risk. 


*FAA  Amendments  27-25  and  29-29  of 
November  13, 1989. 

**FAA  Amendments  21-69, 27-28, 29-32,  and 
91-223. 


Table  12.  Relative  risk  of  serious  injury  (RSI)  in 
Army  helicopters  (Class  A  &  B) 


Type  of  Helicopter 

RSI/ 100,000 
occupant  hours 

UH-60 

5.11 

AH-1 

4.13 

OH-58 

2.91 

UH-1 

1.36 

/ 

Safety  Is  Risk  Management 

To  manage  your  risk,  you  must  first  understand 
your  total  risk.  Prudent  risk  management  will  re¬ 
duce  both  probabilities  in  the  RSI  formula  (probabil¬ 
ity  of  an  accident  times  probability  of  a  serious  in¬ 
jury)  and  achieve  the  lowest  possible  risk.  Accident 
prevention  programs  attempt  to  reduce  the  probabil¬ 
ity  of  an  accident.  Training,  standardization,  equip¬ 
ment,  maintenance,  and  positive  management  atti¬ 
tude  toward  safety  are  key  factors  in  reducing  the 
probability  of  an  accident  occurring.  This  important 
effort  must  continue.  Pre-accident  planning,  flight 
following,  aircraft/occupant  survival  gear  and  train¬ 
ing,  and  aircraft  crashworthiness  features  address 
the  reduction  of  the  probability  of  serious  injury. 
This  important  effort  must  also  continue.  To  believe 
that  you  can  prevent  all  accidents  is  analogous  to  a 
baseball  team  made  up  of  only  a  pitcher  and  a  catch¬ 
er.  The  other  baseball  team  members  will  not  be 
needed,  because  the  pitcher  will  always  strike  out 
the  batter.  Totally  effective  accident  prevention  is  a 
worthwhile  goal,  but  realistically,  it  is  doubtful  that 
it  will  ever  happen.  The  aviation  community  must 
work  to  reduce  both  probabilities. 

UNIQUE  SAFETY  ENVIRONMENTS 

Australian  CAA  Study  of  Single  vs.  Twin  Heli¬ 
copter  Transfer  of  Marine  Pilots 

The  Australian  Civil  Aviation  Authority  (CAA)  con¬ 
ducted  a  study  (Ref.  9)  in  September.  1989  to  respond 
to  a  recommendation  to  mandate  twin-engine  heli¬ 
copters  be  used  rather  than  single-engine  helicop¬ 
ters  for  marine  pilot  transfers.  A  marine  pilot  is  a 
special  ship  pilot  that  boards  the  ship  and  brings 
that  ship  into  a  harbor.  He  likewise  will  pilot  a  ship 
out  of  a  harbor  to  open  sea  after  which  he  is  returned 
to  land.  A  single-engine  helicopter  is  used  for  this 
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transfer  to  and  from  the  ship.  A  recommendation 
had  been  made  to  mandate  the  use  of  only  twin- 
turbine  helicopters.  The  study  looked  at  accident 
data  from  around  the  world.  Applicable  paragraphs 
from  the  study  findings  and  conclusions  are  quoted 
below: 

"The  CAA  believes  that  greater  weight 
should  be  given  to  actual  accident  perfor¬ 
mance  figures  (where  these  are  available) 
than  to  theoretical  assumptions  about  fatal 
accident  rates  derived  from,  say,  engine  shut¬ 
down.  For  example,  it  would  fail  to  account 
for  the  trade-off  between  the  extra  reliability 
from  having  a  second  engine  and  the  lower  re¬ 
liability  of  the  more  complex  helicopter  sys¬ 
tem.  . . .” 

"Informal  advice  from  the  industry  suggest 
that  it  would  approximately  double  the  cost  of 
transferring  marine  pilots  by  helicopter  if 
twin-engine  helicopters  were  made  compul¬ 
sory.  .  . . 

"This  report  does  not  pursue  costing  further 
because  of  the  lack  of  conclusive  evidence  of 
twin-engined  helicopters  leading  to  lower  fa¬ 
tal  accident  rates. . . . 

"Marine  Authorities  have  indicated  that  in 
some  cases  the  higher  cost  of  twin-engined 
helicopters  could  lead  to  them  reverting  to 
launches  to  transfer  pilots,  which  these  au¬ 
thorities  have  stated  is  less  safe  than  transfer 
by  helicopter. . . . 

"CONCLUSION 

"The  CAA  believes  the  proposal  to  regulate  to 
make  it  compulsory  to  use  twin-engined  heli¬ 
copters  for  the  transfer  of  marine  pilots  to  and 
from  ships  should  be  shelved  at  this  time. 
The  CAA  concludes  that  the  proposal  should 
be  shelved  because  the  present  very  low 
engine-failure  accident  rate  is  acceptable, 
and  because  there  is  no  conclusive  evidence 
that  using  twins  would  result  in  a  lower  fatal 
accident  rate.” 

This  Australian  study  is  a  good  example  of  the  im¬ 
portance  of  analyzing  accident  data  for  factual  infor¬ 
mation  considering  all  aspects 


Helicopter  Accidents  at  Elevated  Structures 

The  accident  histories  of  turbine-powered  helicop¬ 
ters  at  elevated  structural  platforms  were  compared 
to  determine  if  the  ICAO  Annex  6  prohibition  of 
single-engine  helicopter  operation  from  elevated 
structures  was  justified.  The  USA  accidents  from 
NTSB  for  1984  through  1988  were  used.  There  were 
no  distinctions  made  between  type  of  operations  be¬ 
ing  conducted  such  as  air  transport  vs  aerial  work. 
Since  the  vast  majority  of  helicopter  uses  are  for  hire 
or  remuneration  in  some  aspect,  it  is  not  possible  to 
use  the  ICAO  definitions.  Many  helicopter  oper¬ 
ations  in  the  USA  are  not  clearly  within  the  ICAO 
definitions  and  also  can  change  categories  of  work 
several  times  in  a  day.  For  example,  a  helicopter 
used  for  emergency  medical  services  (EMS)  can  fall 
in  the  operational  categories  of  business,  unschedu¬ 
led  air  taxi,  and  other  work.  If  the  owner  is  a 
government/  municipality  entity  or  the  civil  opera¬ 
tors  contract  with  a  government  agency  for  helicop¬ 
ter  services,  the  same  helicopter  can  also  be  consid¬ 
ered  to  be  in  the  category  of  "public  use.”  The  acci¬ 
dent  data  should  be  considered  in  its  entirety  to  be 
consistent  with  flight  hours. 

Each  NTSB  helicopter  accident  narrative  for  the 
latest  available  data  (1984  through  1988)  was  used 
to  determine  all  accidents  that  occurred  on  an  ele¬ 
vated  landing  site  or  approaching/departing  the  ele¬ 
vated  structure.  A  key  word  search  was  used  for  the 
following  words  in  the  NTSB  accident  narratives. 
These  key  words  were 


Elevated 

Helipad 

Net 

Structure 

Helideck 

Rail 

Platform 

Heliport 

Pad 

Rig 

Hospital 

Raised 

Roof 

Building 

Deck 

The  resulting  accidents  were  then  separated  into 
movable  landing  structures  or  stationary  landing 
structures.  Accidents  at  movable  landing  structures 
of  landing  dollies,  trailers,  trucks,  boats,  barges,  and 
portable  landing  structures  were  eliminated  as  not 
being  applicable  to  the  safety  history  of  helicopters 
operating  on  an  elevated  structure.  The  stationary 
elevated  structure  accidents  are  those  that  were  at 
rooftops  or  offshore  platforms.  There  were  no  single¬ 
piston  helicopter  accidents  related  to  stationary  ele¬ 
vated  platform  structures,  but  some  were  on  mov¬ 
able  landing  structures. 
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There  were  fifteen  single-turbine  helicopter  acci¬ 
dents  at  stationary  elevated  platform  structures. 
Twelve  were  at  offshore  platforms  and  three  at  a 
rooftop  Of  the  fifteen  accidents,  there  were  four 
power  losses  reported.  There  were  no  material  fail¬ 
ures  found  during  the  investigation  of  two  of  these 
power  losses.  The  remaining  eleven  clearly  resulted 
from  human  causes  as  follows: 

Takeoff  with  aircraft  tied  down 
Landing  gear  caught  on  safety  net 
Landing  gear  caught  on  deck  obstruction 
Main  rotor  blade  strike 

Blown  off  platform  during  engine  start  by  wind 
Elevator  cover  not  removed  prior  to  flight 

There  were  thirteen  twin-turbine  helicopter  acci¬ 
dents  at  elevated  platform  structures.  Nine  were  at 
offshore  platforms  and  four  were  at  rooftops.  Of  the 
nine  offshore  platform  accidents,  two  were  due  to 
material  failures  of  tail  rotor  drive  shafts  and  one 
pylon  mounting  failure  allowing  ground  resonance. 
The  remaining  seven  offshore  platform  accidents 
were  human  caused  as  follows: 

Tail  or  tail  rotor  strike 
Main  rotor  strike 

Flight  controls  restricted  (maintenance  error) 
Takeoff  with  wheel  in  safety  net 
Flight  control  loss 


Of  the  four  rooftop  accidents,  two  were  power  losses 
due  to  fuel  exhaustion.  A  tail  rotor  strike  and  a 
flight  controls  restricted  (loose  object  in  cockpit) 
made  up  the  two  remaining  accident  causes.  Two  of 
these  twin-turbine  helicopter  accidents  on  station¬ 
ary  elevated  structures  were  deleted  prior  to  the  ac¬ 
cident  rate  calculation  as  no  FAA  flight-hours  were 
available  for  the  year  of  the  accidents.  These  acci¬ 
dents  were  two  twin-turbine  SA-330J  helicopters 
which  were  included  above  to  show  the  types  of  acci¬ 
dents  (i.e.,  13  accidents)  but  are  deleted  in  Table  13 
when  accident  rates  are  used  (i.e.,  11  accidents).  All 
single-turbine  accidents  (which  were  Model  206s)  on 
stationary  elevated  structures  were  usable  acci¬ 
dents. 

Table  13  shows  the  USA  elevated  structure  helicop¬ 
ter  accident  history  for  1984  through  1988.  This  ta¬ 
ble  also  identifies  the  stationary  elevated  structure 
accidents  that  were  related  to  power  losses.  For  all 
accidents  at  elevated  structures,  the  accident  rates 
for  the  single-turbine  and  twin-turbine  helicopters 
were  0.21  and  0.76  per  100,000  flight-hours,  respec¬ 
tively.  Thus  the  single-turbine  rate  was  72.4%  low¬ 
er  than  the  rate  for  twin-turbine  helicopters.  Con¬ 
sidering  only  those  related  to  power  losses,  the 
single-turbine  and  twin-turbine  helicopter  accident 
rates  were  0.071  and  0.139  per  100,000  flight-hours, 
respectively.  The  single-turbine  rate  for  power  loss 
accidents  was  48.9%  lower  than  the  twin-turbine 


Table  13.  USA  elevated  structure  turbine  helicopter  accident  history  (1984-1988) 


Type  of  Aircraft 

Fleet  Flight- 
hours 

All  Accidents 

All  Causes 
Rate* 

Power- Loss 
Accidents 

Power-Loss 

Accident 

Rate* 

Single 

7,035,846 

15 

0.21 

5 

0.071 

Twin 

1,442,116 

11 

0.76 

2 

0.139 

Using  hours  of  aircraft 
models  involved  in 
accidents: 

Single 

206 

5,215,001 

15 

0.29 

5 

0.096 

Twin 

222 

932,438 

11 

1.18 

2 

0.214 

AS355 

B0105 

S58T 

S76 

•Accidents  per  100,000  flight-hours 
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rate.  The  second  part  of  Table  10  is  similar,  except 
the  fleet  flight-hours  used  were  for  only  the  models 
that  were  involved  in  elevated  structure  accidents. 
In  this  analysis,  the  single-turbine  and  twin-turbine 
accident  rates  for  all  causes  were  0.29  and  1.18  per 
100,000  flight-hours,  respectively.  The  single¬ 
turbine  rate  was  75.4%  lower  than  the  twin-turbine 
rate.  Considering  the  power  loss  accidents,  the 
single-turbine  and  twin-turbine  accident  rates  are 
0.096  and  0.214  per  100,000  flight  hours,  respective¬ 
ly.  The  single-turbine  rate  for  power-loss  accidents 
was  55.1%  lower  than  the  twin-turbine  rate.  Thus, 
the  actual  helicopter  accident  experience  related  to 
helicopter  operations  at  a  stationary  elevated  struc¬ 
ture  does  not  justify  the  prohibition  of  single-engine 
helicopters. 

Offshore  Helicopter  Operator  Experience 

Petroleum  Helicopters,  Incorporated  (PHI)  is  the 
largest  commercial  helicopter  operator  in  the  world. 
Most  of  their  flying  is  offshore  oil  support  and  as 
such  provides  an  excellent  example  of  safe  helicop¬ 
ter  operations  in  a  difficult  environment.  The  latest 
PHI-furnished  flight-hour  information  and  NTSB 
accident  data  on  PHI  helicopters  from  1984  through 
1988  indicate  that  single-turbine  helicopters  can  be 
and  are  operated  safely  over  water  and  onto  elevated 
platforms.  PHI  flight  hours  in  Table  14  show  that 
66.1%  of  their  flying  was  in  single-turbine  helicop¬ 
ters.  Table  15  compares  the  PHI  accident  rates  for 
all  causes  with  the  U.S.  civil  helicopter  fleet  rates 
for  all  causes.  PHI  accident  rate  for  single-turbine 
helicopters  was  65.8%  and  62.2%  lower  than  the 
general  U.S.  single-turbine  and  twin-turbine  heli¬ 
copter  rates,  respectively.  This  shows  that  a  safe  op¬ 
eration  can  be  and  is  being  conducted  using  single¬ 
turbine  helicopters  without  severe  operational  regu¬ 
lations  like  the  recent  ICAO  Annex  6,  Amendment  1 
change. 


Table  14.  Petroleum  Helicopter,  Inc  (PHI)  flight- 
hours  (1984  through  1988) 


Type  of  Aircraft  Flight-hours 

Percentage  of 
Total 

Single  turbine 

1,064,439 

66.1% 

Twin  turbine 

545,670 

33.9% 

Total 

1,610,117 

100% 

206  only 

982,611 

61.0% 

Table  15.  PHI  vs.  USA  helicopter  accident  rates 
(Accidents  from  NTSB,  Hours  from 
FAA  and  PHI,  84  -  88) 


Type  of  Aircraft 

US 

(NTSB/FAA) 

PHI 

(NTSB/PHI) 

Single  turbine 

5.49 

1.88 

Twin  turbine 

4.37 

1.65 

206  only 

4.28 

1.73 

*  Accidents  per  100,000  flight-hours 
Time  of  Accident,  Day  vs.  Night 


Since  the  actual  flight  hours  flown  at  different  times 
of  the  24-hour  day  are  not  know,  it  is  difficult  to  de¬ 
termine  relative  safety  of  night  flight  vs.  daylight 
flight.  However,  it  is  possible  to  approximate  the 
distribution  of  flying  at  night  by  considering  the 
random  nature  of  material  failures.  For  the  period 
of  1982  through  1988,  the  USA  distribution  of  acci¬ 
dents  (all  causes)  by  the  time  of  day  from  NTSB  data 
is  shown  in  Figure  12.  The  breakpoints  between 
light  and  dark  were  assumed  to  be  0600  and  1959 
hours.  This  distribution  of  accidents  should  be  con¬ 
servative,  as  most  flying  is  done  during  the  summer 
months  when  the  length  of  daylight  is  highest.  This 
indicates  that  91.8%  and  82.8%  of  all  single-turbine 
and  twin-turbine  helicopter  accidents,  respectively, 
occurred  during  daylight  hours.  Figure  13  shows 
the  time  of  accident  distribution  of  airworthiness- 
failure  accidents  (all  material  failures  including  the 
engine).  For  all  airworthiness-failure  accidents, 
98.2%  and  94.1%  of  single-turbine  and  twin-turbine 
helicopter  accidents,  respectively,  are  occurring  in 
daylight  hours.  The  two  figures  have  similar  distri¬ 
bution;  thus  accidents  due  to  material  failures  do 
not  appear  to  be  adversely  affected  by  lighting,  and 


Source:  NTSB  82  -  88  Daily  light 
conditions 


Dark  AM  Daylight  Dark  PM 


Daylight  is  0600  to  19S9  hours 


I-N120 

Fig.  12.  Time  of  accidents  due  to  all  causes. 
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Source:  NTSB  82-88 
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■I  SINGLE  PISTON 
I  I  SINGLE  TUM1NE 

rrm  twin  turbine 


Dark  AM  Daylight  Dark  PM 

Daylight  is  0600  to  1959  hours 

1  N121 

Fig.  13.Time  of  accidents  due  to 
airworthiness  failures. 

therefore,  there  is  no  rationale  to  prohibit  single¬ 
engine  helicopters  from  flying  at  night. 


The  big  difference  in  helicopter  and  fixed-wing  air¬ 
craft  emergency  landings  is  that  the  fixed-wing  air¬ 
craft  requirement  for  a  long  cleared  landing  site  in¬ 
creases  the  likelihood  of  injury  during  the  final 
phase  of  the  emergency.  Conversely,  a  helicopter 
(regardless  of  the  number  of  engines)  can  use  a  land¬ 
ing  site  that  is  quite  small  in  comparison  to  the 
fixed-wing  aircraft  needs.  Likewise,  visibility  at 
night  is  not  as  critical  in  a  helicopter  as  in  a  fixed- 
wing  airplane  due  to  the  helicopter’s  lower  speed 
and  greater  maneuverability  during  autorotation. 


Likelihood  of  Material  Failure  Accident  at 
Night 


Assuming  a  Model  206  and  a  twin-turbine  helicop¬ 
ter  flew  10  hours  of  darkness  every  night  through¬ 
out  one  full  year,  each  helicopter  would  fly  3,652.5 
hours  each  year.  Using  the  NTSB/FAA  accident 
data  for  1984  through  1988  (Table  7),  the  likelihood 
of  an  accident  due  to  a  material  failure  (which  in¬ 
cludes  engine)  for  the  twin-turbine  helicopter  is  esti¬ 
mated  at  once  in  17.2  years  whereas  the  206  likeli¬ 
hood  is  estimated  at  once  in  26.1  years.  Both  of 
these  likelihoods  are  extremely  remote.  Thus  the 
likelihood  of  any  material-failure-caused  accident  is 
51.4%  higher  in  a  twin-turbine  helicopter  than  in 
the  single-turbine  Model  206.  Again,  there  is  no  ra¬ 
tionale  that  supports  the  prohibition  of  night  flights 
of  single-engine  helicopters. 

BELL’S  SAFETY  TRAINING  APPROACH 

Accident  data  analyses  can  be  used  to  determine  if 
safety  programs  or  other  factors  are  making  a 


change  in  the  accident  frequencies.  Two  out  of  three 
accidents  are  not  caused  by  airworthiness  failure 
but  are  basically  due  to  human  error.  This  is  not  a 
"pilot”  problem,  but  a  human  problem  (i.e . ,  the  prob¬ 
lem  is  not  merely  related  to  the  process  of  piloting, 
but  to  the  larger  problem  of  human  limitations).  Ac¬ 
cidents  caused  by  human  error  (generally  called  pi¬ 
lot  error)  are  an  extremely  complex  problem  with  a 
large  number  of  root  causes  and  an  even  larger  num¬ 
ber  of  potential  solutions.  Engineers  and  regulatory 
agencies  are  comfortable  working  on  physical  parts 
as  their  performance  and  failure  modes  are  fairly 
predictable.  Thus  aviation  safety  efforts  in  the  past 
have  made  significant  gains  in  minimizing  airwor¬ 
thiness  failures.  More  attention  is  now  being  made 
toward  understanding  and  eventual  reduction  of  hu¬ 
man  error  accidents.  An  engineering  study  in  1985 
and  1986  into  worldwide  human  error  accidents  of 
Beil  civil  helicopter  models  found  that  poor  judg¬ 
ment  was  the  common  factor  in  all  of  these  accidents 
(Ref.  2).  Two  directions  of  concentrated  effort  at  Bell 
were  launched  in  1987  to  aggressively  attack  the 
complex  human  error  problem,  with  the  emphasis 
on  J  udgment  Training. 

Individual  Judgment  Training  Aid 

Human  Factors  Engineering’s  approach  was  to  de¬ 
velop  an  artificial-intelligence  based  software  which 
would  allow  a  pilot  to  use  a  personal  computer  (PC) 
as  a  judgment  (decision-making)  simulator.  This  is 
roughly  a  decision-making  simulator  equivalent  of 
the  present-day  six-axis  motion  simulators  that  al¬ 
low  the  pilot  to  test  his  motor  skills  without  endan¬ 
gering  his  aircraft  or  his  life.  This  program,  called 
Cockpit  Emergency  Procedures  Expert  Trainer  (CE- 
PET),  also  includes  emergency  procedures  training. 
A  CEPET  was  developed  for  the  Bell  JetRanger 
(206BIII)  and  LongRanger  (206L-3),  with  one  for  the 
212/412  completed  late  in  1990.  The  CEPET  is  a 
long-term  effort  where  an  individual  pilot  can  use 
the  CEPET  software  and  a  PC  to  improve  his  safe 
decision-making  skills.  A  CEPET  package  is  pro¬ 
vided  with  each  new  aircraft  delivery  starting  in 
1991.  Pilots  can  also  purchase  a  separate  CEPET 
package. 

Group  Safety  Training 

The  other  direction  was  concentrated  safety  educa¬ 
tion.  System  Safety  Engineering  developed  a  3-hour 
safety  briefing  for  immediate  use  with  groups  of  pi¬ 
lots/managers.  This  safety  briefing  presented  by  the 
Chief  Safety  Engineer  includes  how  to  measure 
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one’s  risk,  what  happens  in  a  crash,  how  one  can  im¬ 
prove  his  chances  of  survival,  causes  of  accidents, 
root  causes  of  human  error,  and  Judgment  Training. 
Judgment  Training  emphasizes  the  use  of  all  re¬ 
sources  available  to  the  pilot  and  is  something  of  a 
single-pilot  version  of  the  Cockpit  Resource  Man¬ 
agement  (CRM)  used  in  crew-served  airplanes. 
Judgment  Training  emphasis  is  on  situational 
awareness  and  internal  pilot  monitoring  rather 
than  crew  interactions  of  CRM.  Judgment  Training 
is  also  called  Pilot  Decision  Making  (PDM)  and 
Aeronautical  Decision  Making  (ADM).  Portions  of 
the  FAA  study,  DOT/FAA/PM-86/45,  Aeronautical 
Decision  Making  for  Helicopter  Pilots  (Ref.  10)  are 
used  in  this  safety  briefing  and  the  FAA  report  is 
given  to  the  student  for  further  self  study.  This  safe¬ 
ty  brief  is  given  at  operator’s  and  regional  safety 
seminars  and  is  included  in  Bell’s  weekly  206  pilot’s 
ground  school  as  part  of  the  Helicopter  Professional 
Pilots  Safety  (HELIPROPS)  program. 

Bell’s  Chief  Training  Pilot  also  conducts  customer 
HELIPROPS  safety  briefings  on  safety  awareness, 
professionalism,  and  management’s  role  in  safety. 
These  safety  briefings  are  held  at  Bell,  customer 
sites,  and  regional  safety  seminars.  In  1988,  Bell’s 
Customer  Support  and  Service  Department  (CSSD) 
initiated  the  HELIPROPS  program  to  add  continu¬ 
ity  and  coordination  of  these  safety  education  ef¬ 
forts.  A  HELIPROPS  Administrator  was  assigned 
full  time  for  coordination  and  to  also  conduct  cus¬ 
tomer  site  and  regional  safety  seminars.  The 


HELIPROPS  effort  was  spread  to  the  othr  nt dicop¬ 
ter  manufacturers  with  three  companies  u  i;  ed  in 
the  techniques  that  were  working  for  Bell.  These 
companies  then  started  their  own  safety  training 
version  of  HELIPROPS. 

The  worldwide  effects  of  this  4-year  safety  education 
effort  on  the  human  error  accident  rate  since  the 
Model  206  effort  was  fielded  in  1987  is  shown  in  Ta¬ 
ble  16.  There  have  been  over  5,000  Model  206  series 
helicopters  produced  or  70%  of  Bell’s  entire  civil  tur¬ 
bine  helicopter  model  fleet.  Bel!  also  conducts  pilot 
flight  training  in  Model  206s.  Based  on  these  two 
factors,  the  concentrated  safety  education  effort  has 
been  directed  at  Model  206  pilots.  For  comparison, 
the  same  worldwide  data  for  Bell’s  medium  civil  he¬ 
licopters  models  (i.e.,  204B,  205A1,  214B,  212, 
214ST,  222,  and  412)  are  also  shown  in  Table  16. 
These  medium  helicopter  data  indicate  some  reduc¬ 
tions  in  human  error  causes  but  were  offset  with 
non-human-error  causes;  thus  the  accident  rate  for 
all  causes  was  basically  the  same  over  the  two  4- 
year  periods.  Conversely,  accident  rates  due  to  hu¬ 
man  error  in  a  206  for  the  4-year  period  before  the 
initiation  of  this  safety  effort  (1983-1986)  and  the 
four-year  period  since  (1987-1990),  show  a  36.2%  re¬ 
duction.  This  is  a  significant  safety  improvement 
since  we  have  covered  only  a  portion  of  all  Model  206 
pilots  in  the  world  thus  far.  The  overall  (all  causes) 
Model  206  accident  rate  is  now  reduced  by  26.3%. 
Since  many  pilots  fly  helicopters  in  addition  to  the 
Bell  Model  206,  we  can  expect  some  spillover  of  the 


Table  16.  Worldwide  Bell  turbine  accident  rates  (Rates  per  100,000  flight-hours) 


Causes  of  Accidents 

Aircraft  and  Period 

Flight-hours 

Human  Error 

Non-Human  and 
Unknown 

All  Causes 

Model  206 

1983  - 1986 

7,903,072 

3.90 

2.05 

5.95 

1987  - 1990 

9,341,573 

2.49 

1.89 

4.38 

Percent  change 

-36.2% 

-7.8% 

-26.3% 

Bell  Mediums 

1983  - 1986 

2,438,515 

2.62 

2.01 

4.63 

1987-1990 

2,472,091 

2.31 

2.39 

4.69 

Percent  change 

-11.8% 

+  18.9% 

+  1.3% 
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Table  17.  USA  human  error  accidents  involving  weather 


Single  Piston 

Single  Turbine 

Twin  Turbine 

206  Single 
Turbine 

Flight-hours 

84-86 

1,899,081 

4,167,156 

821,679 

2,997,911 

87-88 

1,062,171 

2,868,690 

620,437 

2,217,090 

HE  WX  Accidents 
84-86 

26 

40 

7 

25 

87-88 

8 

8 

2 

5 

HE  WX  Accidents 
per  100,000 
flight-hours 

84-86 

1.37 

0.96 

0.85 

0.83 

87-88 

0.75 

0.28 

0.32 

0.23 

HE  WX  Rate 

-45.3% 

-70.8% 

-62.4% 

-72.3% 

Reduction 

beneficial  effects  of  Judgment  Training  (ADM), 
which  should  affect  the  overall  helicopter-industry 
accident  rate.  Further,  since  the  Model  206  flies 
most  of  the  helicopter  fleet  hours,  the  industry  acci¬ 
dent  rates  will  be  lower. 

Analysis  of  human  error  accidents  involving  weath¬ 
er  shows  a  changing  trend  in  the  USA.  NTSB  acci¬ 
dent  data  and  FAA  flight  hours  for  1984  through 
1988  were  divided,  with  an  early  period  of  1984 
through  1986  compared  to  the  later  period  of  1987 
and  1988.  The  results  are  Table  17.  The  year  1987 
was  the  beginning  of  Bell’s  concentrated  safety 
training  programs  to  reduce  human  error  accidents 
as  discussed  above.  Thus  the  range  of  human  error 
accident  rate  reductions  due  to  poor  weather  deci¬ 
sions  in  the  most  recent  time  period  has  been  signifi¬ 
cantly  reduced  between  45%  and  72%.  This  reduc¬ 
tion  is  due  to  safety  training,  not  mandatory  regula¬ 
tions. 

The  annual  human  error  accident  rates  in  the  Model 
206  were  determined  for  the  period  of  1982  to  mid- 
1991  to  check  the  statistical  significance  at  even 
longer  periods.  The  FAA  flight-hours  for  1982 
through  1989  were  used.  The  flight-hours  for  1990 
and  1991  (through  June  30)  were  forecast,  using  the 
trend  of  the  previous  8  years  of  FAA  data.  The  acci¬ 
dents  used  were  from  NTSB  data  (1982  through 
1988,  the  latest  available).  The  accidents  occurring 
from  1989  through  June  30,  1991  were  estimated 
from  Bell  information.  Figure  14  shows  the  Model 
206  accident  rates  due  to  human  error  for  the 


accident  data  period  1982  through  1987  (i.e.,  prior  to 
the  introduction  of  safety  training)  and  the  period 
1987  through  June  30,  1991  (with  concentrated 
Model  206  safety  training,  including  ADM).  The 
human  error  accident  rate  for  the  ADM  safety  train¬ 
ing  period  ( since  1987)  is  significantly  different  from 
the  previous  period  95  times  out  of  100.  This  reduc¬ 
tion  in  accident  rate  has  occurred  with  no  changes  to 
regulatory  restrictions.  As  a  further  check,  the  acci¬ 
dent  rate  of  the  Model  206  for  all  causes  was  deter¬ 
mined  with  and  without  the  ADM  safety  training, 
as  shown  in  Fig.  15.  Curve  A  is  the  actual  accident 
rate  for  the  Model  206,  with  the  ADM  training  ef¬ 
fects  since  1987.  Curve  B  is  the  estimated  accident 
rate  for  the  Model  206  with  a  continuation  of  the 
consistent  human  error  accident  rate  trend  of  1982 
through  1986  extended  throughout  the  remaining 
years.  The  trend  of  the  accident  rates  without  ADM 
is  consistent  with  historical  accident  rates.  The  ac¬ 
tual  accident  rates  with  ADM  safety  training  (Curve 
A)  are  significantly  different  from  that  normally  ex¬ 
pected  without  ADM  training  (Curve  B)  to  a  signifi¬ 
cance  level  of  0.05.  In  other  words,  95  times  out  of 
100,  the  two  curves  are  significantly  different. 

The  Canadian  government  is  starting  to  integrate 
PDM  into  their  pilot  training  requirements  as  of 
1991.  PHI,  the  largest  U.S.  helicopter  operator,  in¬ 
troduced  Judgment  Training  as  an  integral  part  of 
their  internal  training  which  has  subsequently  cut 
their  accident  rates  in  half.  Bell  looks  for  further 
accident  rate  reductions  as  we  continue  this  worth¬ 
while  effort.  Judgment  Training  (eg.,  PDM  or 
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IN  122 

Fig.  14. Model  206  human  error  accident  rates 
(USA,  1982  to  mid- 1991) 


Fig.  15.  Model  206  accident  rate  with  and 

without  ADM  (USA,  1982  to  mid-1991). 


ADM)  has  more  safety  improvement  potential  that 
the  total  elimination  of  all  airworthiness  failure 
causes  (a  primary  goal  since  the  start  of  aviation). 

Consideration  of  the  safety  education  effects  of  sev¬ 
eral  manufacturers  efforts  on  the  human  error  acci¬ 
dent  rates  of  the  types  of  helicopters  in  the  USA  is 
found  in  Table  18.  This  shows  a  significant  reduc¬ 
tion  in  human  error  accident  rates  in  the  turbine  he¬ 
licopter  fleet.  More  work  is  needed  in  the  single¬ 
piston  fleet.  Since  safety  education  is  an  ongoing  ef¬ 
fort,  it  will  take  several  years  to  reach  all  helicopter 
pilots. 


CONCLUSIONS 

Helicopters  are  not  fixed-wing  aircraft  and  therefore 
behave  differently  when  undergoing  any  engine 
failure.  The  helicopter’s  ability  to  autorotate  allows 
a  low  speed  emergency  landing  from  an  engine 
failure  and  the  selection  of  suitable  landing  sites. 
One  should  not  make  safety  decisions  on  any  one 


Table  18.  Safety  education  effects  on  human 
error  accident  rates  NTSB/FAA 
(USA-registered) 


Type  of 
Helicopter 

Rate* 

Before 

(84-86) 

Rate* 
Since 
(87  &  88) 

Percent 

Changes 

Single  piston 

11.16 

10.92 

-2.2% 

Non-206  single 
turbine 

4.11 

3.07 

-25.3% 

Twin  turbine 

2.56 

1.61 

-37.1% 

206  single 
turbine** 

3.40 

1.76 

-48.2% 

•Human  error  accidents  per  100,000  hours 
•‘Concentrated  HELIPROPS  safety  education 


helicopter  part  without  considering  the  safety 
aspects  of  ail  other  parts  and  the  human  causes. 
Considering  all  airworthiness  failures  (all  material 
failures  including  the  engine),  the  twin- turbine 
helicopter  accident  rate  is  1.5  times  higher  than  in 
the  single-turbine  206.  Considering  all  accident 
causes,  the  twin-turbine  helicopter  accident  rate  is 
close  to,  but  still  higher  than,  the  Model  206  rate. 
Mandating  twin  engines  does  not  reduce  the 
likelihood  of  a  material-failure-caused  accident,  but 
merely  changes  the  types  of  failures  that  cause 
accidents.  Single-turbine  accident  experience 
related  to  elevated  structures  is  better  than  for  twin 
turbines.  The  risk  to  the  neighborhood  around  a 
heliport  from  an  airworthiness-failure-caused 
accident  is  lower  for  the  single-turbine  Model  206 
than  for  twin-turbine  helicopters.  Mandatory  use  of 
twin-engine  helicopters  around  the  world  does  not 
make  sense  from  a  safety  point  of  view.  In  some 
specific  harsh  environments  such  as  the  North  Sea, 
the  twin-turbine  helicopter  is,  and  should  be,  used. 
However,  there  are  many  environments  and  uses 
where  the  twin-turbine  helicopter  is  not  the  best 
choice. 

Based  on  the  preceding  analyses ,  there  are  no 
statistically  significant  differences  between 
Performance  Class  2  (twin-turbine  engine)  and 
Performance  Class  3  (single-turbine  engine)  accident 
rates,  and  therefore  the  restrictions  placed  on 
Performance  Class  3  operations  are  unwarranted 
from  a  safety  standpoint.  Additionally,  these 
restrictions  can  impose  severe  humanitarian  and 
economic  hardships  by  denying  the  less  costly 
services  that  could  be  provided  by  a  simpler  and  less 
restrictive  Performance  Class  3  single-engine  heli¬ 
copter. 
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The  safety  measurement  method  that  should  be 
used  is  strictly  determined  by  the  subject  of  primary 
concern  The  denominator  of  the  frequency  rate  will 
include  this  primary  concern.  If  aircraft  damage 
frequency  is  your  primary  concern,  then  an  accident 
per  aircraft  flight-hour  method  is  appropriate.  If  the 
mission  is  the  primary  concern,  then  the  accidents 
per  mission  (e  g.,  launch,  departure,  takeoff,  flight, 
trip,  passenger  mile  or  patient  transport)  method  is 
appropriate.  If  the  primary  concern  is  the  risk  of  an 
accident  in  a  neighborhood  without  regard  to  the 
aircraft  occupants,  then  years-between-accidents 
measurement  for  that  specific  neighborhood  expo¬ 
sure  is  appropriate.  With  the  safety  of  the  aircraft 
occupant  as  the  primary  concern,  measuring  rela¬ 
tive  risk  of  serious  injury  per  occupant  flight-hour  is 
the  best  method. 

The  recent  concentrated  Judgment  Train- 
ing/ADM/PDM  efforts  of  manufacturers,  operators, 
and  regulatory  agencies  have  made  a  significant  re¬ 
duction  in  human  error  accidents.  This  major  reduc¬ 
tion  has  occurred  without  any  regulatory  changes  or 
limitations.  Major  improvements  in  helicopter  safe¬ 
ty  for  the  future  require  the  continuation  and  refine¬ 
ment  of  these  safety  efforts.  Occupant  risk  in  a  heli¬ 
copter  is  low  now  but  the  aviation  community  can, 
and  must,  reduce  it  further. 
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PROPOSED  ACTION  PLAN  TO  IMPROVE  ADM  EFFECTIVENESS 


1.0  INTRODUCTION 

ADM  training  benefits  in  terms  of  reducing  human  error  accident  rates  were 
documented  during  the  presentations  at  the  workshop.  Basic  research  needs, 
training  implementation  problems  and  the  need  for  additional  modeling  work  were 
also  identified.  This  document  provides  the  aviation  industry  and  the  FAA  with  a 
suggested  roadmap  to  assist  in  the  development  of  improved  ADM  concepts  and 
training  methods.  The  basic  questions  that  will  be  addressed  are: 

•  Can  decision  making  tasks  be  identified  and  defined? 

•  What  are  the  training  objectives? 

•  What  are  the  appropriate  training  strategies? 

•  How  can  the  training  effectiveness  be  evaluated? 

2.0  GOALS 

The  central  goal  of  this  proposed  plan  is  to  provide  a  coordinated  workplan  which 
can  be  used  to  make  the  significant  step  from  current  ADM  training  to  an  enhanced, 
integrated  ADM/CRM/AQP  flight  training  concept.  Although  ADM  is  only  one 
part  of  this  overall  system,  it  has  been  identified  by  the  participants  (operators, 
regulators  and  researchers)  as  the  part  which  currently  needs  attention  and 
development  work.  This  goal  supports  the  user's  desire  to  avoid  "add-on"  material 
which  increases  training  time  and  cost. 

3.0  NEEDS 

The  proposed  research  supports  the  FAA's  National  Plan  for  Aviation  Human 
Factors  (NPAHF).  The  seventh  objective  of  the  NPAHF  is:  "to  develop  enhanced 
methods  of  training  and  selection  for  aviation  system  personnel".  The  NPAHF 
recognizes  the  need  for  realistic  training  in  crew  coordination,  judgment  and 
decision  making  in  flight  operations.  The  development  of  the  proposed  ADM 
training  methods  and  models  will  provide  critical  input  to  specific  "Flightdeck 
Environment"  requirements  of  the  National  Plan  in  three  areas: 

1.  The  development  and  demonstration  of  an  intelligent  "human  centered" 
advanced  technology  flightdeck. 

2.  The  development  of  computer-based  models  of  flightcrew  decision  making  and 
other  cognitive  processes  to  serve  as  the  basis  for  the  design  of  new  systems. 

3.  The  development  of  new  technologies  for  flightcrew  performance  measurement 
to  answer  questions  about  in-flight  performance  of  both  individual  pilots  and  crews. 
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4.0  PROPOSED  ACTION  PLAN 


This  detailed  Action  Plan  presents  the  work  necessary  to  develop  advanced  ADM 
training  programs.  It  includes  tasks  which  should  be  started  as  early  as  possible 
because  of  the  need  for  basic  information  as  recommended  by  the  participants  at  the 
workshop  .  It  also  includes  a  presentation  of  all  subtasks  necessary,  as  a  minimum, 
to  address  the  cognitive  and  decision  demands  required  by  different  aircraft  types, 
different  missions  and  pilots/crews  with  different  levels  of  experience. 

Sections  6.3,  7.3  and  8.3  of  Volume  I  of  this  report  provided  technical 
recommendations  from  the  participants  for  future  research  and  development 
requirements  from  commercial,  military  and  general  aviation  perspectives.  These 
recommendations  were  used  along  with  the  Key  Concepts  (Section  5.0)  and 
Conclusions  (Section  9.0)  of  Volume  I  as  a  basis  for  developing  the  following  tasks. 

Task  1  —  Defining  the  Structure  of  Decision  Making  Tasks 
Task  2  —  Developing  Training  Requirements 
Task  3  --  Specifying  Training  Strategies 
Task  4  --  Evaluating  Training  Effectiveness 

The  basic  work  to  be  performed;  the  expected  outputs  and  the  flow  of  information 
are  represented  in  Figure  1.  An  overview  description  of  the  four  primary  Action 
Plan  tasks  follows  the  figure.  The  specific  analytical  elements  and  subtask  details  of 
each  task  are  described  in  Sections  4.1, 4.2, 4.3  and  4.4. 


TASK  1 


TASK  2 


TASKS 


TASKS 


Figure  1  ADM  TRAINING  PROGRAM  DEVELOPMENT  TASKS 
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As  shown  in  the  figure,  the  first  task  is  comprised  of  developing  an  ADM  specific 
taxonomy  and  performing  a  comprehensive  operational  analysis  of  cognitive  task 
demands  (termed  a  Cognitive  Task  Analysis  -  CTA-  by  the  participants).  The 
primary  outputs  of  Task  1  are  expected  to  be  the  identification  and  definition  of 
generalized  ADM  methods  and  models  including:  decision  types,  decision  making 
styles  and  decision  making  capabilities  (novice  vs.  expert).  These  outputs  will  be 
refined  to  formulate  and  develop  ADM  specific  models  for  use  throughout  the 
training  development  process.  The  understanding  and  knowledge  gained  during 
the  performance  of  Task  1  will  also  be  used  to  specify  preliminary  training 
objectives. 

Task  2  will  begin  the  development  of  training  requirements  for  the  various  training 
environments.  These  will  include:  aircraft  type  (airplane  vs.  helicopter  and 
automated  vs.  non-automated  flightdeck);  type  of  operation  (mission);  single  pilot 
vs.  multi-crew;  and,  level  of  expertise  (novice  to  expert).  As  proposed  at  the 
workshop,  the  initial  work  in  developing  training  requirements  might  be  limited  to 
high  and  low  workload  missions  such  as  the  airline,  air  transport  mission  (low 
workload)  and  offshore,  helicopter,  single  pilot  IFR  mission  (high  workload).  The 
spectrum  of  experience  from  ab  initio  to  ATP  rated  pilots  should  eventually  be 
evaluated  from  a  cognitive  training  needs  viewpoint.  However,  this  analysis  could 
begin  with  the  novice  vs.  recurrent  training  requirements  specification  since  these 
are  the  two  largest  target  audiences  and  also  represent  two  extremes  of  the  ADM 
spectrum.  The  outputs  from  this  task  are  expected  to  be  a  refined  set  of  training 
objectives.  The  analysis  can  then  proceed  to  develop  lesson  plans  and  manuals  to  be 
used  in  an  evaluation  of  alternative  training  strategies. 

Task  3  will  involve  tailoring  the  training  strategies  to  the  spectrum  of  candidate 
training  audiences.  The  different  needs  of  general  aviation,  the  airlines  and  the 
military  will  be  considered.  The  large  differences  in  availability  of  training  facilities 
for  centralized  vs.  de-centralized  audiences  (i.  e.,  military  vs.  private  pilots)  and  the 
affordability  of  the  training  will  be  the  primary  considerations  in  developing 
alternative  materials,  methods  and  tools.  Expected  outputs  include  individual 
study  and  self  test  manuals,  context  based  classroom  instruction,  interactive  training 
devices  (computer  based  and  interactive  video)  and  simulator  scenarios  suitable  for 
the  LOFT  environment. 

Task  4  will  evaluate  the  ADM  taxonomy,  models,  training  objectives,  and  training 
strategies  in  an  integrated  Training  Effectiveness  program.  The  scope  and  duration 
of  this  program  is  difficult  to  speculate  on  at  this  time.  However,  each  training  test 
plan  will  require  the  use  of  cognitive  and  skill  performance  measures.  These  are 
currently  referred  to  as  Behavioral  and  Technical  "markers"  in  the  CRM  Advisory 
Circular.  These  markers  require  further  R.D.T.  &  E.  themselves,  but  some  type  of 
objective  measures  will  be  required  to  adequately  assess  whether  or  not  the  ADM 
training  is  operationally  meaningful.  The  outputs  from  this  task  are  expected  to  be 
refined  training  plans,  strategies  and  methods.  There  will  be  some  iteration 
between  the  initial  evaluation  outputs  and  each  of  the  first  three  tasks. 
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4.1  Task  1  -  Defining  the  Structure  of  Decision  Making 

The  basic  work  required  in  Task  1  of  the  analysis  was  generated  from  the  workshop 
participants’  expressed  need  for  a  better  understanding  and  definition  of 
aeronautical  decision  making.  This  includes  the  immediate  need  to  begin  R  &  D  on 
a  basic  classification  tayonomy  for  aeronautical  decision  making.  The  taxonomy 
development  should  be  done  in  parallel  with  a  detailed  analysis  of  cognitive  task 
demands  by  phase  of  flight.  These  definitional  analyses  would  include  both  single 
pilot  general  aviation  and  commercial  air  transport  mission  or  flight  scenarios. 

They  would  also  explicitly  consider  the  differences  between  helicopter  and  airplane 
workloads  and  decision  making  demands.  Figure  2  illustrates  the  steps  involved  in 
this  task. 


GENERAL 


Figure  2  DEFINING  GENERAL  ADM  METHODS  AND  MODELS 

4.1.1  Technical  Subtasks 


As  shown  in  Figure  1,  the  development  of  an  ADM  specific  taxonomy  and 
classification  schema  will  include  five  subtasks: 

1.  Identifying  the  elements  of  a  good  decision  (normal,  emergency  and  novel 
situations  should  be  considered) 

2.  Identifying,  defining  and  developing  types  of  decisions  (initial  group  efforts 
identified  Binary,  Rule  Based  and  Adaptive  types.  See  Volume  I  Section  8.3) 

3.  Identifying  decisional  styles  under  various  situations  and  task  demands 

4.  Defining  characteristics  of  expert  decision  makers 

5.  Identifying  individual  vs.  team  decisional  processes  (single  pilot  vs.  crew) 

The  Cognitive  Task  Analysis  will  require  selection  of  operational  environments  to 
be  analyzed  (e.  g.,  air  transport  operations)  and  performing  a  detailed  task  demand 
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analysis  for  each  flight  phase  (takeoff,  climb,  departure  enroute  climb,  cruise,  etc.). 
The  decision  making  demands  for  each  phase  should  be  analyzed  for  complexity 
(high,  medium,  low)  and  the  most  complex  decisional  demands  prioritized  for 
more  detailed  analyses  of  the  effects  of  expertise  (novice  vs.  expert)  and 
environment  (automated  vs.  non-automated  flighdeck). 

The  results  of  the  taxonomy  and  CTA  analyses  will  be  used  to  characterize  situation 
specific  Aeronautical  Decision  Making.  The  spectrum  from  classical  analytical 
decision  making  to  the  more  recent  naturalistic  decision  making  paradigms  will  be 
analyzed  for  applicability.  Appropriate  ADM  methods  will  be  defined  within  this 
spectrum  as  a  function  of  cognitive  task  demands.  The  goal  of  this  analysis  is  to 
produce  a  compendium  of  tailored  ADM  methodologies  and  models  for  the  variety 
of  task  demands  and  environments  typically  encountered  by  each  type  of  operator. 
From  this  understanding  of  the  decision  making  processes  across  environments, 
advanced  ADM  training  materials,  methods  and  models  will  be  developed. 

4.1.2  Applicable  Resources 

The  performance  of  the  proposed  task  should  capitalize  to  the  greatest  extent 
possible  on  related  research  which  has  been  completed  or  is  still  underway. 
Participants  at  the  Workshop  from  the  FAA,  NASA,  NTSC,  the  airlines  and 
research  companies  are  all  currently  involved  in  directly  related  basic  research.  In 
addition,  the  University  of  Colorado  at  Boulder  (a  participant),  the  University  of 
Texas,  the  University  of  Illinois,  Massey  University  in  New  Zealand  and  others  are 
actively  developing  and  testing  the  types  of  methods  and  tools  which  may  need  to 
be  integrated  in  the  proposed  task.  What  is  needed  is  the  establishment  of  an  ADM 
development  project  with  centralized  oversight  and  a  core  group  of  Subject  Matter 
Experts  (SMEs).  This  will  help  ensure  the  achievement  of  all  specific  objectives. 

The  oversight  should  be  in  the  form  of  a  Technical  Monitor  who  would  act  as  an 
integrator  and  facilitator  between  other  research  efforts  as  well  as  establish  the 
statement  of  need  for  additional  research  to  support  the  tasks  described.  The  FAA  is 
the  logical  agency  to  provide  this  function  due  to  the  fact  that  it  is  the  only  central 
agency  that  has  supported  and  directed  research  in  ADM,  CRM,  EDM  and  AQP.  The 
FAA  Technical  Monitor  could,  perhaps,  be  supported  by  Technical  Representatives 
from  NASA,  NTSC,  USAF,  etc.  Table  1  was  prepared  to  illustrate  the  responsibility 
of  the  Technical  Monitor  and  indicates  the  FAA  in  this  role  as  an  example.  The 
areas  of  known  expertise  and  expressed  interest/capabilities  are  indicated  as 
applicable  resources  for  each  subtask.  This  analysis  may  be  incomplete,  but  it  is 
important  as  a  first  step  in  illiciting  comments  and  action  on  the  proposed  work. 

4.1.3  Proposed  Scope 

A  one  year  initial  project  effort  is  proposed  culminating  with  a  second  ADM 
workshop  to  report  on  research  findings,  status  and  future  directions.  A  one  to 
three  year  continuing  effort  may  be  required  to  attain  the  desired  general  ADM 
methodology  and  tailored  models,  training  objectives  and  select  target  groups. 
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Table  1  RESEARCH  RESOURCES  AVAILABLE  FOR  DEFINING  THE 
STRUCTURE  OF  ADM 
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4.2  Task  2  —  Developing  Training  Requirements 

This  task  will  begin  with  the  general  decision  making  methods  and  models 
identified  in  Task  1  and  develop  alternative  training  strategies  for  a  variety  of  user 
operations  based  upon  realistic  context  and  cues.  The  realism  will  be  developed 
from  specific  analyses  of  relevant  situations  of  varying  workload  for  a  number  of 
aircraft  types  and  for  varying  levels  of  pilot/crew  expertise.  As  specified  by  the 
workshop  participants  (Section  7.3  B  Volume  I),  this  task  will  provide  information 
on  decision  making  when  crews  have  multiple  tasking  and  provide  guidance  on 
how  training  can  be  developed  for  different  levels  of  an  aviator's  career.  Figure  3 
illustrates  the  subtasks  required  to  support  this  analysis  and  their  relationship. 


TRAINING  CONTEXT 
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4.2.1  Technical  Subtasks 


As  shown  in  Figure  3,  the  development  of  tailored  ADM  training  objectives  and 
plans  requires  consideration  of  the  following: 

1.  Applying  the  general  methods  and  models  from  Task  1  to  the  operational 
environment  and  situation  (i.e.,  context)  to  be  trained. 

2.  Specifying  the  preliminary  cognitive  training  objectives  and  the  type  of  ADM  to 
be  trained. 

3.  Developing  the  training  context  and  cue  environment  (A/C  type,  workload, 
procedures,  level  of  automation,  etc. 

4.  Defining  specific  training  requirements  and  objectives. 

5.  Developing  lesson  plans  and  simulator  scenarios  as  required  to  attain  objectives. 

The  heavy  dashed  lines  in  Figure  3  provide  one  example  of  the  training 
requirements  development  process.  As  indicated,  a  narrow  bodied,  air  transport 
aircraft  with  an  automated  flightdeck  is  selected  as  the  training  environment.  A 
long  haul  mission  with  normal  procedures  is  the  specified  workload  or  task 
demand.  This  training  session  will  involve  a  mult-crew  circumstance  because  of 
the  aircraft  type  selected.  The  target  crew  would  be  undergoing  recurrency  training. 
Based  on  these  parameters,  specific  training  requirements  in  terms  of  session 
objectives,  a  lesson  plan  for  the  crew  to  exercise  their  ADM  capabilities,  the 
instructors  training  syllabi,  and  if  desired,  a  simulator  scenario  (an  alternate  would 
be  a  classroom  or  video  tape  training  session). 

The  results  of  Task  1  could  be  used  repeatedly  in  the  manner  illustrated  in  this 
example  to  generate  a  complete  set  of  ADM  training  and  evaluation  lessons  for  each 
of  the  desired  Training  Contexts.  Again,  the  participants  suggested  that  the  initial 
emphasis  be  on  low  workload  air  transport  operations  and  high  workload  helicopter 
single  pilot  IFR  operations  as  two  extremes.  The  goal  of  Task  2  is  to  develop 
Training  Requirements  that  can  be  used  to  evaluate  various  training  strategies  as 
well  as  a  range  of  ADM  skills.  The  performance  of  this  task  will  require  several 
iterations  and  refinements  for  each  operation  type  (mission)  and  workload  level 
desired.  However,  completion  of  this  task  for  each  combination  should  result  in  a 
reasonable  number  of  usable,  realistic  training  requirements  by  operation  type  (e.g., 
short-haul  air  transport). 
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4.2.2  Applicable  Resources 

The  development  of  meaningful  training  objectives  will  require  a  cooperative  and 
iterative  effort  between  government  and  industry  resources.  However,  the  general 
flow  of  this  development  is  from  NASA,  University  and  Corporate  researchers  to 
the  Air  Force,  Navy,  Airline  and  General  aviation  user  communities.  Therefore, 
Table  2  was  prepared  as  a  strawman  resource  application  exercise.  As  shown  in  the 
table,  the  FAA  Technical  Monitor  would  rely  on  Subject  Matter  Experts  (SMEs)  from 
the  other  user  communities  throughout  this  task.  NASA  would  be  expected  to  take 
the  early  lead  in  applying  the  generalized  ADM  methodology  and  models  developed 
in  Task  1.  NASA  could  use  University  and  Corporate  decision  making  researcher's 
as  resources  throughout  subtasks  1,  2,  and  3.  However,  the  user  communities 
(AFSC,  NTSC  and  the  Airlines)  would  become  heavily  involved  during  the 
development  of  the  training  context  options  of  subtasks  3,  4,  5  and  the 
University /Corporate  resources  become  less-and-less  involved  as  the  training 
requirements  development  emphasis  switched  from  applying  theories  and  models 
to  developing  realistic  training  contexts  and  scenarios.  As  in  Task  1,  exceptions  to 
this  general  flow  of  work  would  be  made  based  upon  expressed  interest, 
demonstrated  capabilities  and  expertise. 

Table  2  RESEARCH  RESOURCES  AVAILABLE  FOR  DEVELOPING  TRAINING 
REQUIREMENTS 
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4.2.3  Proposed  Scope 

The  Training  Requirements  task  could  actually  begin  during  the  last  six  months  or  3 
months  of  the  Task  1  effort.  As  in  Task  1,  a  one  year  initial  effort  is  proposed  as  a 
trial  period.  It  is  recommended  that  a  small  number  of  training  plans  and  scenarios 
be  targeted  for  completion  in  the  first  six  months  as  a  calibration  on  progress, 
problems  and  possible  redirection.  After  the  initial  12  months,  the  outputs  should 
include  training  requirements  specification  (objectives  &  plans)  for  at  least  one  user 
from  each  of  the  three  communities  --  airline,  military  and  general  aviation. 
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43  Task  3  —  Specifying  Training  Strategies 


The  work  required  in  this  task  will  focus  on  applying  various  training  strategies  and 
alternatives  to  the  outputs  from  Task  2.  That  is,  the  training  requirements  will  be 
transformed  into  meaningful,  cost  effective  training  materials,  tools  and  techniques. 
The  alternatives  to  be  considered  were  summarized  in  Volume  I  Section  8.3  but  are 
repeated  in  Table  3  for  ease  of  reference  . 

Table  3  ADVANCED  ADM  TRAINING  TOOLS  AND  TECHNIQUES 


JTYPE  OF  TRAINING 

MATERIALS  AND  METHODS 

A.  Individual  Study 

1.  Manuals  and  Self-tests 

2.  Other  ADM  publications 

B.  Training  Syllabi 

1.  Student 

2.  Instructor 

3.  Evaluator 

C.  Context  Based 

1.  Classroom  Vignettes  (structured  hangar  flying) 

2.  Library  of  Videos 

D.  Experience  Transfer 

1 .  Grand  Rounds  (Mentor)  concept 

2.  Peer  Training  (alternate  instructor  roles) 

E.  Activity  Based 

1.  Computer  Based  Training  (expert  system) 

2.  Interactive  video  disk 

F.  Simulator  Scenarios 

1.  Normal  procedures  (based  on  CTA) 

2.  Emergency  and  Novel  situations 

The  application  of  each  of  these  alternatives  would  require  analysis  of  the  needs  and 
capabilities  of  the  user  communities.  Figure  4  depicts  the  process  involved. 


TRAINING  STRATEGIES 


DECISION  TYPE 

V  AVOIDING  RISK 

V  ANDCIPATmG  OCMANOS 

V  RESPONDING  TO  DEMANDS 


INDIVIDUAL  STUOY 
1.  ManuaH 
2..  Acttvfty  (CBT) 


CLASSROOM  •  INSTRUCTOR 

1.  Training  SyttabJ 

2.  Contort  Stand 

EXPERIENCE  TRANSFER 

1.  Manioc  Coneapt 

2.  Paar  Training 

SIMULATOR  SCENARIOS 

1.  Normal  Procadurti 

2.  Emargancy  Procoduraa 
2.  Nova!  Situations 


Figure  4  ANALYZING  APPROPRIATE  TRAINING  STRATEGIES 
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4.3.1  Technical  Subtasks 


As  shown  in  Figure  4,  the  basic  process  required  to  select  appropriate  training 
alternatives  has  one  primary  and  two  secondary  subtasks.  First,  the  Target  User 
Group  must  be  defined:  General  Aviation,  Airlines  or  Military.  Then  the  selection 
of  D«rision  Type  to  be  trained  and  the  level  of  Expertise  of  the  trainee  group  defined. 
At  this  point,  the  alternative  training  objectives  and  plans  from  Task  2  can  be 
reviewed  vs.  the  various  implementation  strategies  from  Table  3.  The  primary 
consideration  for  selecting  the  appropriate  training  strategy  (methods  and  tools) 
should  be  the  ability  to  train-to-objective  at  minimum  cost  in  terms  of  both  training 
time  and  initial  investment.  Obviously  there  is  a  broad  range  of  capabilities 
between  the  user  groups  that  will  influence  the  strategy  selected.  The  primary 
outputs  of  this  analysis  should  be  the  specification  of  training  strategies  suitable  for 
all  three  user  groups.  Tailored  training  manuals,  classroom  vignettes,  interactive 
videos,  computer  based  training,  and  simulator  scenarios  should  be  developed  for 
evaluation  in  the  next  task. 

4.3.2  Applicable  Resources 

At  this  point  in  the  development  of  advanced  ADM  training  methods  the  available 
resources  will  have  to  be  selected  and  used  based  upon  the  specific  strategies  which 
are  to  be  developed  and  evaluated.  That  is,  it  is  difficult  to  speculatively  specify 
resources  for  unknown  strategies.  However,  past  experience  and  available 
capabilities  suggest  that  the  FAA  might  take  the  lead  in  developing  general  aviation 
and  helicopter  training  strategies.  These  may  include  selections  from  Table  3 
training  types  A.,  B.,  C.,  and  E.  The  airlines  and  NASA  may  choose  to  work 
independently  or  in  consort  to  develop  the  more  complex  and  costly  strategies  in 
training  types  D.  and  F.  as  well  as  considering  applications  of  the  other  strategies  for 
the  more  sophisticated  users.  Precise  selection  and  tasking  of  resources  will  require 
further  consideration  prior  to  completion  of  this  task. 

4.3.3  Proposed  Scope 

The  scor  e  of  this  effort  is  somewhat  evolutionary  in  nature.  However,  if  realistic 
training  requirements  are  developed  in  Task  2  for  a  limited  set  of  target  groups,  then 
some  advanced  training  strategies  (manuals,  videos,  CBT  devices,  and  simulator 
scenarios)  should  be  able  to  be  developed  within  two  years  of  the  concept 
specification.  Based  upon  the  postulated  scopes  of  Tasks  1  and  2,  this  would  mean 
some  limited  strategies  could  be  fielded  in  two  and  one-half  to  three  and  one-half 
years  after  initiation  of  Task  1. 

4.4  Task  4  —  Evaluating  Training  Effectiveness 

This  task  will  evaluate  the  ADM  taxonomy,  models,  training  objectives  and  training 
strategies  developed  in  the  three  previous  tasks.  The  basic  work  required  was 
generated  from  the  participants  strong  recommendation  regarding  the  importance 
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of  the  validation  step.  The  specific  design  of  the  work  performed  in  this  task  will 
depend  upon  the  number  and  types  of  training  strategies  developed  in  Task  3  as  will 
the  duration  and  cost  of  this  task.  However,  three  basic  subtasks  and  the  overall 
scope  of  this  task  is  illustrated  in  Figure  5.  The  general  type  of  work  involved  for 
evaluating  each  training  module  is  described  in  the  next  section. 


INPUTS  FROM 
TASK  3 


Figure  5  EVALUATING  TRAINING  EFFECTIVENESS  OF  ADVANCED 
ADM  STRATEGIES 

4.4.1  Technical  Subtasks 

The  performance  of  each  training  effectiveness  evaluation  will  require  three 
subtasks  as  a  minimum: 

1.  Test  plan  development 

2.  Test  performance  and  data  collection 

3.  Data  analysis  and  development  of  conclusions/recommendations. 
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The  test  plan  development  will  include  experiment  design,  resource  specification 
test  group  selection,  pilot/crew  subject  selection,  data  collection  specification  and 
data  analysis  technique  proposals.  The  resource  specification  will  entail  a 
coordinated  evaluation  of  the  specific  user  communities  to  be  evaluated  and  the 
typical  training  resources  they  would  use.  The  analysis  would  then  proceed  to 
match  experimental  resource  capabilities  to  meet  those  typical  needs.  For  example. 
Flight  Safety  International  may  be  used  to  evaluate  the  training  effectiveness  of  new 
ADM  training  scenarios  for  multi-crew,  corporate  helicopter  operators  since  they  are 
a  typical  resource  for  that  user  group. 

The  test  performance,  data  collection  and  data  analysis  subtasks  would  also  require  a 
coordinated  industry /government  effort.  NASA,  military  and  FAA  personnel  and 
facilities  would  be  required  to  support  the  effectiveness  evaluation  tests.  In 
addition.  University  and  industry  research  teams  would  be  required  to  provide 
members  of  the  analysis  team. 

4.4.2  Applicable  Resources 

The  performance  of  the  required  training  effectiveness  evaluation  should  utilize,  to 
the  greatest  extent  possible,  civil  and  military  simulator  and  training  facilities.  The 
test  subject  pilots  should  also  include  both  civil  and  military  pilots  at  the 
appropriate  experience  levels.  There  are  two  reasons  for  this  strategy.  First,  the 
government  facilities  offer  access  to  sophisticated  facilities  with  data  collection  and 
information  not  otherwise  available.  Second,  the  results  of  training  strategy 
development  research  using  these  resources,  although  not  focused  solely  on  the 
commercial  cockpit,  may  provide  information  on  decision  making  when  the  crews 
have  multiple  tasking  (which  is  standard  for  military  aircrews)  and  may  give 
guidance  on  how  training  can  be  developed  for  different  levels  of  an  aviator's 
career.  In  addition,  the  captive  group  of  subjects  in  a  highly  controlled 
environment  offers  an  initial  opportunity  to  test  the  viability  of  the  training  in  a 
timely  manner.  Table  4  provides  initial  guidance  on  the  use  of  these  resources 
along  with  industry  and  university  support  for  the  evaluation  of  the  training 
strategy  alternatives. 

Table  4  RESOURCES  AVAILABLE  FOR  EVALUATING  TRAINING 
EFFECTIVENESS 


GOVERNMENT 
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AIRLINES 
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2.  Testing  and  Data  Collection 

mum 
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V 
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4.4.3  Proposed  Scope 


Test  design  and  facility  evaluation/selection  could  begin  as  soon  as  prototype 
training  strategies  have  been  developed  for  the  three  audiences:  airlines,  military 
and  general  aviation.  This  could  occur  approximately  three  and  one-half  years  after 
the  start  of  the  Advanced  ADM  Training  project.  However,  due  to  the  other 
commitments  of  both  facilities  and  personnel  involved,  the  detailed  availability, 
schedule  and  use  of  these  facilities  will  require  a  much  closer  examination  as  the 
need  develops.  Estimates  of  a  one  to  two  year  data  collection  and  analysis  period 
were  made  for  the  training  effectiveness  evaluation  by  the  participants  at  the 
workshop.  This  would  have  completion  of  the  evaluation  between  four  and  one- 
half  and  five  years  after  the  start  of  the  project. 

5.0  PROPOSED  OVERALL  SCHEDULE 

The  following  overall  schedule  was  compiled  to  provide  a  snapshot  of  the  task 
duration  and  overlap  envisioned  at  this  time.  It  is  provided  as  guidance  for 
discussion  and  comment.  The  actual  generation  of  a  more  meaningful  schedule 
will  require  actual  interest  in,  and  initiation  of,  the  proposed  Action  Plan  by  the  key 
participants  indicated. 
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Figure  6  PROPOSED  OVERALL  SCHEDULE 
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